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STUDIES IN LANGUAGE BEHAVIOR 


I. A PROGRAM OF RESEARCH 


WENDELL JOHNSON 
University of Iowa 


HE STUDIES by Fairbanks (5), Mann 
pee and Chotlos (4), which are 
presented in this issue of The Psycho- 
logical Monographs, constitute the be- 
ginnings of a program of research in 
language behavior. They have been com- 
pleted in the order named. The present 
paper is intended as an introduction to 
them, and to the general program which 
they represent. 

The importance of language and of 
symbolization generally, as a distinctly 
human form of behavior and as a basic 
factor in personal and social problems, 
is generally recognized (9, 10, 12, 20). 
The effective scientific investigation of 
such behavior, however, depends upon 
the development of highly reliable and 
differentiating measures, by means of 
which specified aspects of language be- 
havior might be systematically observed 
in relation to one another and to other 
variables. With such measures, signifi- 
cant testable hypotheses can be formu- 
lated and checked, and a body of de- 
pendable information can be accumu- 
lated. 


SPECIFIC OBJECTIVES 


The proposed program of research is 
designed to: 


1. Develop reliable and_ differentiating 
measures of specified aspects of language 
behavior. 

2. Determine the degree to which the re- 
sulting measures are intercorrelated. 

3. Determine the degree of correlation be- 
tween these measures and such other 
pertinent variables as those involved in 
environmental influences, physiological 
conditions, intelligence, and personality 
adjustment. 


4. Apply the measures to a comprehensive 

investigation of language development. 

5. Determine the degree to which language 

behavior, as measured, is modifiable un- 
der specified conditions. 

6. Determine the degree to which modifica- 
tion in language behavior is associated 
with modifications in other aspects of 
behavior or adjustment. 

. Indicate the normal characteristics of 
language development and language be- 
havior, and the varieties of disorder or 
abnormality in such behavior, in terms 
of the measures used. 


~I 


Types. of Language Measures to Be 
Investigated 

No attempt will be made here to pres- 
ent a review of the theoretical and ex- 
perimental literature dealing with the 
problems with which this program is 
concerned. It is sufficient to say that 
previous work in the field has suggested 
many of the procedures to be employed, 
and that others have been suggested by 
preliminary research carried out by the 
writer, or under his direction. A com- 
prehensive review of language behavior 
studies has been published by Sanford 
(12). The following types of language 
measures are to be investigated: 

Type-token ratio (TTR). This is a 
measure of vocabulary “flexibility” or 
variability, designed to indicate certain 
aspects of language adequacy. It ex- 
presses the ratio of different words 
(types) to total words (tokens) in a given 
language sample. If in speaking 100 
words (tokens) an individual uses 64 
different words (types), his TTR would 
be .64. In order to develop the most 
highly reliable and differentiating form 
of the TTR, it is to be computed for 
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given language samples in the following 
various ways: 

a. For all words spoken or written by 
a given individual, or in a given lan- 
guage sample, and separately for words 
representing the various grammatical 
categories; for words in different fre- 
quency categories—for example, the 500 
most frequently used words, the next 
500 most frequently used words, etc., 
as determined by the published word- 
counts of Thorndike (17), Horn (6), and 
others, or by the word-counts to be de- 
rived from the present investigations; 
etc. 

b. With varying statistical or mathe- 
matical procedures, thus: 

The over-all TTR, as computed for 
an entire language sample. T'T'R’s for 
samples of different magnitudes are not 
directly comparable because of the tend- 
ency for the TTR to vary inversely with 
size of sample. A knowledge of the pre- 
cise character of this inverse relationship 
might make it possible to compare di- 
rectly I°TR’s for samples differing in 
length, by means of a correction table. 
The feasibility of constructing such a 
table is to be investigated. The study by 
Chotlos (4) throws considerable light on 
this problem. 

The mean segmental TTR. TTR’s for 
samples of different magnitudes can be 
made comparable by dividing each 
sample into like-sized segments of, say, 
100 words each, computing the TTR 
for each segment and then averaging the 
segmental TITR’s for each sample. It 
can be safely assumed that such seg- 
mental ITTR’s are directly comparable, 
so long as they represent segments of 
equal size, and that means of such seg- 
mental ‘I°TR’s are also directly com- 
parable. Results obtained by using seg- 
ments of different magnitudes—as 100- 
word segments, 500-word segments, etc.— 
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are to be compared, in order to ascer- 
tain the size of segments that will allow 
for the most reliable and differentiating 
mean segmental TTR. The above men- 
tioned study by Chotlos (4) is concerned 
with this problem also. 

The cumulative TTR curve. A curve 
of the cumulative TTR for a given lan- 
guage sample can be plotted by com- 
puting successive I’T'R’s as increments 
are added to the sample. For instance, 
the cumulative TTR for a 1000-word 
sample would be plotted as follows: 
TTR values are to be represented along 
the ordinate and number of words along 
the abscissa. The abscissa values may be 
in units of one word, or ten words, or 
100 words, etc., as desired. If the unit 
is one word, 1000 TTR’s would be com- 
puted in plotting the cumulative curve 
for the 1000-word sample; if the unit is 
ten words, 100 IT R’s would be com- 
puted; if the unit is 100 words, ten 
TTR’s would be computed, etc. Thus, 
if the unit is ten words, the first value 
will represent the TTR for the first ten 
words of the sample, the second will 
represent the TTR for the first 20 words, 
the third will represent the TTR for the 
first 30 words, etc. The problem of fitting 
an equation to the resulting curve is 
dealt with in some detail by Chotlos (4). 
Basically, the problem concerns the rela- 
tion between D (number of different 
words, or types) and N (number of 
words, tokens) in the given sample. This 
problem has been given considerable at- 
tention by Zipf (20), Carroll (3), and 
Skinner (16). The relevant data presented 
by Chotlos (4) indicate the degree to 
which the relation of D to N promises a 
means of predicting vocabulary, in the 
sense that the value of D for a given N 
provides a basis for predicting D for a 
specified N of larger magnitude. 

The decremental TTR curve. Sup- 
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pose a 1000-word sample to be divided 
into ten 100-word segments. The TTR 
is computed for the first segment. Then, 
the number of different words in the 
second segment that did not occur in 
the first segment—i.e., the number of 
new types introduced in the second seg- 
ment—is found. The TTR for the sec- 
ond segment is then computed by di- 
viding this number—not the number of 
types, but the number of new types—by 
100, which is the number of tokens in 
the second segment. In the same way, the 
TTR’s for the third, fourth, and each 
of the other segments may be computed, 
by dividing the number of tokens, 100 
in each case, into the number of new 
types introduced into the sample for the 
first time in the segments under con- 
sideration. The resulting curve of these 
successive segmental TTR’s may be ex- 
pected to show a relatively steeper slope 
than the cumulative TTR curve, and 
the measure representing the slope of 
this curve may be found to be of special 
interest. It represents, of course, the rate 
of decrement in the use of new types, 
the rate at which the individual ‘‘uses 
up” his vocabulary in producing a lan- 
guage sample. Decremental T’I'R’s 
should represent in a peculiarly direct 
quantitative manner one aspect of lan- 
guage development, when applied to 
language samples secured successively 
from the same children. The decre- 
mental TTR curve is, of course, the first 
derivative of the cumulative TTR curve, 
and thus it is not actually necessary to 
fit a curve to the decremental TTR data 
if the cumulative TTR curve has been 
computed. 

Type-frequencies. A simple objective 
language measure is that which expresses 
the frequency of occurrence of each dif- 
ferent word, or type. Such frequencies, 
as reported for large samples of written 


language by Thorndike (17), Horn (6), 
and others, have been used chiefly in the 
preparation of school readers, spelling 
books, etc. Certain other uses of such 
data are obvious. When type-frequencies 
are based on the kinds of language 
samples to be used in the present pro- 
gram they may be regarded as represent- 
ing language behavior norms. In previ- 
ous studies of word-frequencies it would 
seem that the primary objective has been 
simply to determine the relative fre- 
quency of occurrence of each word, and 
with some exceptions special interest has 
attached to those words which have been 
found to occur with especially high 
frequencies. The main objective of the 
present program in this connection is 
somewhat different. Chief interest lies 
in ascertaining individual and group dif- 
ferences in the relative frequency with 
which particular kinds of words are used. 
One may determine (a) type-frequency 
changes that characterize language de- 
velopment; (b) type-frequency character- 
istics of the language of special groups, 
especially those that may be found to dif- 
ferentiate one group from another, as 
schizophrenics from normal subjects, 
scientists from novelists, etc.; (c) the par- 
ticular type-frequencies that correlate 
significantly with such other variables 
as intelligence, emotional stability, edu- 
cational level, etc. Attention may be 
given to the following types of words 
(and to any others that may be found to 
be useful): 


a. Self-reference words. 

b. Quantifying terms (precise numerical 
words). 

c. Pseudo-quantifying terms (words loose- 
ly indicative of amount, size, etc., such 
as much, many, lots; or very, highly, etc., 
used as qualifiers of other pseudo-quan- 
tifying terms, as in such expressions as 
“very much’’). 

d. “Allness” terms (superlative or extreme 
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words, such as never, always, all, no- 
body, everyone, etc.). 

e. Words expressive of negative evalua- 
tion, such as no, don’t, etc., and horrid, 
unsatisfactory, dislike, etc. 

f. Words expressive of positive evalua- 
tion. 

g. Qualification terms (words that serve to 
qualify or limit statements, such as_ ex- 
cept, but, however, tf, etc.). 

h. Terms indicative of consciousness of ab- 
stracting (such words as apparently, 
seems, appears, as if, to me, etc.; as in- 
dicated by the last two examples, for 
purposes of this type of analysis it will 
be necessary to treat certain phrases as 
single words. What we call the dogmatic 
or “closed mind” attitude might be ex- 
pected to be characterized by language 
in which these terms are relatively lack- 


ing.) 


Ratios of any one of the above types 
of words to any one of the other types 
might be computed for given language 
samples, and their significance evaluated. 
The ratio of the terms indicative of con- 
sciousness of abstracting to “allness” 
words, for example, might be expected 
to differentiate individuals and groups 
in ways that should be of theoretical and 
practical importance in the study of per- 
sonality.! 

The relative frequency of use of the 
various grammatical types of words— 
nouns, adjectives, verbs, adverbs, etc.— 
might also be determined, as well as 
ratios of nouns to adjectives, adjectives 
to verbs, verbs to adverbs, nouns to verbs, 
adjectives to adverbs, etc., and the ratio 
of these four to all other words. With 
language development, the relative fre- 
quency of nouns particularly and also 
of verbs may be expected to decrease 


‘Reference made here to “allness” terms and 
to the notion of consciousness of abstracting im- 
plies the writer’s debt to Alfred Korzybski. See 
especially Korzybski, A., Science and Sanity, An 
Introduction to Non-Aristotelian Systems and 
General Semantics, Lancaster, Pa.: Science Press, 
second edition, 1941. 
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with reference to the relative frequency 
of adjectives and adverbs. The degree to 
which these and other possible relation- 
ships can.be utilized as measures of lan- 
guage development and of individual 
and group differences should be ascer- 
tained. Busemann (2) and Boder (1) have 
employed the adjective-verb quotient to 
indicate certain kinds of personality dif- 
ferences, and to differentiate samples of 
written language. Sanford (13) has re- 
ported a personality study involving this 
and other related measures, The present 
series of studies involves analyses in this 
general connection. Mann (11) applies 
the adjective-verb quotient and also ad- 
jective-noun and adverb-verb quotients 
in her comparative study of the written 
language of schizophrenic patients and 
university freshmen. Fairbanks (5) inves- 
tigates the relative frequencies of occur- 
rence of various parts of speech in com- 
paring the spoken language of schizo- 
phrenic patients and university fresh- 
men. Chotlos (4) presents similar data 
in terms of types and tokens, respectively, 
and he also presents ‘I°'TR values for 
nouns, verbs, adjectives and adverbs, re- 
spectively, for written language samples 
obtained from Iowa school children. 
Proportionate vocabulary. How many 
different words or types make up 25, or 
50, or 75 per cent of a given language 
sample? In the study by Fairbanks (5), 
30,000-word samples of spoken language 
were obtained from schizophrenic pa- 
tients and “superior” university fresh- 
men, respectively. For the freshmen just 
46 different words or types comprised 50 
per cent of the 30,000-word sample, and 
for the schizophrenic patients this figure 
was 33 types. This is the more striking, 
perhaps, when expressed by saying that 
for the schizophrenic patients approxi- 
mately one-tenth of one per cent of the 
words made up 50 per cent of the total 
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sample. In fact, one word, the one most 
frequently used by the schizophrenics, 
which was the word J made up slightly 
over 8.3 per cent of their entire 30,000 
words. 

A sample of, say, 1000 words might be 
analyzed in such a way as to yield a 
curve as follows: Along the ordinate per- 
centages would be represented; these 
percentages would correspond to num- 
bers of tokens. For example, suppose 
that 100 tokens make up 10 per cent of 
the 1000-word sample; it is this 10 per 
cent and other percentage values so com- 
puted that would be represented along 
the ordinate. Other percentages would 
lie along the abscissa; these percentages 
would correspond to numbers of types. 
Thus, suppose that 10 types comprise 1 
per cent of the total of 1000 tokens; 
this 1 per cent and other percentage 
values so computed would be represented 
along the abscissa. The curve showing 
the relation between these two sets of 
percentages would be made up of points 
expressing such values as the one cited 
above: for the schizophrenic patients 
0.1 per cent of the words (this percentage 
representing types) made- up 50.0 per 
cent of the sample (this percentage repre- 
senting tokens). The relation symbolized 
by this curve can be expressed mathe- 
matically, of course, and it is proposed 
to examine its usefulness as a basis for 
comparing different language samples 
or any given sample with a norm or 
standard sample. The relationship dis- 
cussed here can be expressed, of course, 
in terms of rank and frequency. That is, 
a curve that is fitted to word-frequencies 
as a function of rank, the most fréquent 
word having the lowest rank number, 1, 
represents in an alternative way the same 
phenomenon that is discussed here in 
terms of proportionate vocabulary. (See 
Zipf [20].) 


5 


Standard frequency vocabulary. The 
word counts that have been published 
by previous workers, and the one to be 
done in the present program, can be 
used separately or pooled in arriving 
at a standard frequency-of-use rank num- 
ber for each different word included in 
them. Such rank numbers would repre- 
sent the relative frequency with which 
each word had been used in the total 
language sample—presumably drawn 
from a more or less representative popu- 
lation of individuals—not in terms of 
the actual number of times each word 
was used, but in terms of its rank. Thus, 
the most frequently used word would 
have a rank number of 1, the next most 
frequently used word would have a rank 
number 2, etc. 

With the resulting table of rank num- 
bers, it would be possible to score any 
given language sample by noting the 
rank number of each word (token) con- 
tained in it, and computing the mean 
(or median) of these rank numbers. ‘The 
lower the mean of the sample the more 
heavily loaded it is with words that are 
used relatively frequently by people gen- 
erally. We may say, then, that this mean 
rank number of a language sample rep- 
resents the “standard frequency vocabu- 
lary” employed in it. It is to be reason- 
ably expected that language develop- 
ment would be characterized by increase 
in this measure, and that the measure 
would serve to differentiate individuals 
and groups. 

A less refined, and perhaps nearly as 
adequate, form of this measure could 
be worked out in terms of standard fre- 
quency rank numbers on a categorical 
basis. That is, the first 100 most fre- 
quently used words, for example, could 
all be given the same rank number, the 
number 1, the second 100 words could 
all be assigned the rank number 2, etc. 
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Statistical analysis may indicate advan- 
tages in classes or categories of unequal 
magnitudes, putting the more frequently 
used words in smaller groups, for in- 
stance, and the less frequently used words 
in larger groups, or vice versa, perhaps 
varying the number of words in a group 
in some relation to the frequency with 
which they are used. Comparison of re- 
sults obtained from use of various forms 
of the measure will determine the rela- 
tive merits of each. 

Verbal output. A very simple language 
behavior measure is that which expresses 
the verbal output of an individual. Indi- 
vidual differences and intra-individual 
variations with respect to verbal output 
are, of course, obvious. Their significance 
in relation to the various aspects of per- 
sonal and social adjustment have not 
been thoroughly or systematically in- 
vestigated. It is planned to include an 
attempt in this direction in the present 
research program. 

Verbal output is not meant to be 
synonymous with speaking or reading 
rate, as that term is used to refer to 
verbal output under relatively optimal 
conditions. An individual’s verbal out- 
put under various conditions may, and 
usually does, fall considerably under 
what it is when he speaks at or near his 
optimal steady rate. Verbal output may 
be expressed, of course, in terms of rate. 

The measure may express number of 
words spoken or written per unit of 
time, or in response to a specified stimu- 
lus under standard conditions. It may 
also express the proportion of a time 
unit during which an individual pro- 
duces spoken or written language. For 
example, two individuals could be com- 
pared by placing them together for one 
hour and recording (a) the speaking 
time of each, (b) the total number of 


words spoken by each, and (c) the verbal 
output of each in terms of words spoken 
per minute. It is to be noted that these 
measures are different from a measure of 
the rate of verbal output while speaking. 
It would be of interest, of course, to 
correlate such a measure of rate with the 
other verbal output measures. 

Word length. Since the studies of Zipf 
(20) have shown word length to be highly 
correlated negatively with frequency of 
use—the shorter the word the more fre- 
quently it occurs—it is not planned at 
this time that measures of word length 
will be included to any important de- 
gree in the present program. It is men- 
tioned here, however, because the data 
to be utilized will be so tabulated that 
word length could be studied if findings 
indicate that this would be advisable. 
It is a rigorously objective and highly 
reliable measure (15). 

Sentence length. Sentence length is a 
measure that presents serious operational 
difficulties in the study of spoken lan- 
guage, although it may be generally 
Satisfactory in the analysis of written 
language. It is planned to include it in 
the analysis of at least a selected set of 
the written language samples. 


SPECIAL TYPES OF LANGUAGE 
BEHAVIOR TESTS 


The Extensional Agreement Index 
(EAI) expresses the degree of agreement 
among 7 persons in defining a given term 
extensionally—i.e., by pointing to or ex- 
hibiting somehow the actual objects, 
phenomena, etc. to which the term re- 
fers.2 Thus, the kind of behavior which 
the EAI is designed to measure is not 
observation so much as word-fact relat- 


?This measure was introduced and briefly 
discussed in Johnson, W., Language and Speech 
Hygiene, referred to in footnote No. 1. 
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ing. The EAI may range in numerical 
value from 0.0 to 1.0, 0.0 representing no 
agreement and 1.0 representing maxi- 
mum possible agreement among n per- 
sons in relating or applying a given 
word as a label to actualities. Its theoreti- 
cal and practical significance lies in the 
fact that it makes possible not only an 
index of a person’s conformity or idio- 
syncrasy in his extensional use of words, 
but also a measure of the degree to which 
any given term may be regarded as 
testable, or extensional, or operational— 
or vague. If in the statement “Stutterers 
are psychoneurotic” the term “psycho- 
neurotic” has an EAI of, say, .18, the 
statement is not to be regarded as highly 
testable or factually meaningful, since 
n persons would disagree considerably 
as to just what is to be observed in order 
that the validity of the statement might 
be tested. The EAI offers, therefore, a 
means of quantifying to some degree 
such notions as are represented by the 
terms “verifiable,” “operational,” etc. 
The EAI may be computed in several 
different ways. Tuthill (18) in a study 
made as part of the present program 
demonstrated a variety of ways of com- 
puting such a measure of extensional 
agreement. The basic formula is 


x 
EAI = — 
y 


in which x represents the number of 
obtained agreements and y the maxi- 
mum possible number of agreements. 
The EAI, then, represents the per cent 
of the maximum possible number of 
agreements that are obtained in a given 
case, 

For example, imagine four different 
pictures and ten different persons who 
are each asked to apply the label “most 


artistic” to one of them. Suppose the 
label is applied to picture A by 3 per- 
sons, to picture B by none, to picture C 
by 5, and to picture D by 2. If there had 
been perfect agreement, all 10 persons 
would have applied the label to the 
same picture. Thus, the number of agree- 
ments among the 10 persons that would 
have occurred under these conditions is 
to be regarded as the maximum possible 
number of agreements. This number 
may be determined by the formula 
(n — 1) .5n and since n = 10, the maxi- 
mum possible number of agreements is 
9 X 5 = 45. The number of agreements 
actually obtained is to be computed as 
follows: The three persons who applied 
the label to picture A agreed 3 times, 
since when n=g8, (n—1) .5n= 3. 
There were no agreements with regard 
to picture B, in terms of the technique 
for computing the EAI that is here be- 
ing used. Using the formula (n — 1) .5n, 
there were 10 agreements in the labeling 
of picture C, and one in the labeling of 
picture D. In all, then, 14 agreements 
were obtained. Therefore, EAI = 14/45 
= .31, which may be interpreted as indi- 
cating that the number of agreements 
obtained was 31 per cent of the maxi- 
mum possible number. 

This is an example of an extremely 
simple case, used to illustrate the appli- 
cation of the basic formula. Another 
example will serve to indicate an im- 
portant modification of the basic for- 
mula. On July g, 1939, the American In- 
stitute of Public Opinion released to 
newspapers the results of a survey in 
which each of several thousand persons 
had been asked to apply one of the 
labels, “Conservative,” “Liberal,” and 
“Radical,” to each .of ten prominent 
Americans.’ The results were presented 


* The material upon which the present discus- 
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in percentages as follows: 


Conservative Liberal Radical 

» J A A 
Hopkins { 55 41 
Roosevelt 1 62 37 
La Guardia 8 64 28 
Farley 13 63 24 
Dewey 15 47 8 
Hull 5 46 3 
Garner 64 32 4 
Vandenberg 67 29 4 
Taft 86 13 1 
Hoover g2 5 3 


These figures represent only the label- 
ing reactions of persons “‘who knew or 
had some idea of the terms when later 
in the survey they were asked point- 
blank what the words .. . meant.” From 
these data it is possible to compute an 
EAI for each of the three terms involved. 
The procedure to be used will differ in 
three important respects from that used 
in the above example of the four pic- 
tures. In the first place, in the first ex- 
ample there was only one label to be 
applied by each of ten persons to only 
one of four possible referents. In the 
present case, there were three labels, any 
one of which was to be applied to each 
of ten referents. In the second place, 
there were ten labelers in the first ex- 
ample; in this one there were many 
thousands, and the numbers have been 
converted into percentages. These per- 
centages will be used instead of the raw 
numbers in computing the EAI’s. In 
(n — 1) .5n, n will represent 100 in com- 
puting the maximum possible number of 
agreements. Lastly, instead of assuming, 
as was done in the first example, that 
agreements occur only when labels are 
applied, and not when they are not 
applied, we shall assume that both the 
application of a label and the refusal to 
apply it may involve agreement. When 
this assumption is made, the net number 





sion is based appeared under the copyright of 
the American Institute of Public Opinion in the 
Des Moines Register, July 9, 1939. 
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of agreements involved in the applica- 
tion and non-application of a given 
label to a given referent can be computed 
as follows. Let x = the number who ap- 
ply the label, and n — x the number who 
do not apply it. Then, the number of 
agreements among those who do apply 
the label is found by the formula, (x —1) 
.5x. Similarly, the number of agree- 
ments among those who do not apply the 
label equals (n — x — 1) .5 (n — x). The 
net number of agreements is found sim- 
ply by subtracting the smaller of these 
values from the larger. And the EAT is 
found by dividing this met number of 
agreements by the maximum possible 
number of agreements. Thus, 


(mn — X — 1) (M — X) — (X — 1) X 


2 2 
EAI — cei az 
(n—t1)n | n 








9 


In this way, the EAI of each given 
term is computed for each referent, and 
the EAI’s of the term for the various 
referents (in the present case, 10) are 
averaged. For the term “Liberal” the 
following results were obtained: 


“LIBERAL” 

| 2x — n| 

“ Labeling | 

[a | 
Hopkins 55 10 
Roosevelt 62 24 
La Guardia 64 28 
Farley 63 .26 
Dewey 47 06 
Hull 46 08 
Garner 32 36 
Vandenberg 29 42 
Taft 13 74 
Hoover 5 .go 
Ave. EAT 34 


The obtained number of agreements 
was, on the average, only 34 per cent of 
the number representing complete agree- 
ment as to the extensional meaning of 
the word “Liberal,” as applied or not, to 
the ten men listed, by the presumably 
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random sample of persons surveyed by 
the Gallup organization in the summer 
of 1939. The variability is of interest. 
As applied to Hopkins, Dewey and Hull, 
the term “Liberal” proved to be almost 
entirely meaningless; there was virtually 
no agreement as to whether these men 
were or were not suitable referents of 
the term. There was relatively high 
agreement, on the other hand, that Taft 
and Hoover were not to be labeled 
“Liberal.”” The mean EAI was .58 for 
“Conservative” and .69 for “Radical.” 
Dr. Gallup, under whose name the 
survey report appeared in the press, did 
not, of course, report his findings in 
terms of these EAI’s. Moreover, he seems 
to have missed a basic point, in stating 
that the survey results indicated “the 
way American voters—rightly or wrongly 
—are classifying the figures in United 
States political life.” (The italics are the 
present writer’s.) The words “rightly or 
wrongly” seem to imply the assumption 
that there is a “right” way and a “wrong” 
way to apply such a label as “Liberal,” 
that such a term has somehow an in- 
trinsic “meaning,” presumably known by 
some means to someone somewhere, 
quite aside from and more valid than 
the extensional meanings ascribed to it 
by those persons who actively relate it 
to various referents. There would ap- 
pear to bé, from an extensional point of 
view at least, no “right” or “wrong” 
about it, except in the sense that in mat- 
ters of this kind one might (or might not) 
prefer to assume that the majority is 
“right.” Be that as it may, however, Dr. 
Gallup carried out, in this particular 
survey, what amounted to a very am- 
bitious effort to determine by vote the 
extensional meanings of a group of 
words. And by using his results to com- 
pute the EAI’s of these words, it becomes 
possible to measure fairly precisely the 


vagueness or factual meaningfulness of 
some of our important political terms. 

The resulting EAI’s afford a degree 
of insight into the processes of political 
controversy, and point to one of the 
fundamental problems in connection 
with social organization. The EAI of .34 
for “Liberal” strongly suggests that such 
a statement as, “America should (or 
should not) have a liberal in the White 
House,” is to be regarded as essentially 
“lyrical.” Like our remarks about the 
weather, which are not to be mistaken 
for meteorological reports, the remark 
that “So-and-so is a liberal” is not to be 
regarded as a statement chiefly descrip- 
tive of So-and-so. For the most part, it 
merely serves to announce one of the 
ways in which the speaker proposes to 
apply the word “Liberal,” and thus it is 
mainly indicative of an aspect of the 
speaker's language behavior. To know 
the EAI of a word, as computed from 
data as adequate as those provided by 
Dr. Gallup, is to know something quite 
precise and significant about the lan- 
guage behavior of a speaker or writer 
who uses it, particularly if he gives no 
indication of awareness of the word's 
descriptive limitations, as these are im- 
plied by its EAI. The descriptive limita- 
tions of a word with an EAI of .34 are 
probably so great as to render it practi- 
cally meaningless referentially in many 
contexts. It is to be regarded as being 
in many instances little more than noise 
or ink marks, meaningful chiefly in be- 
ing symptomatic of the speaker’s or 
writer’s neurosemantic state. That is to 
say, it is more revealing as behavior than 
as language; it symbolizes the speaker 
more than it symbolizes anything he may 
appear to be speaking about. 

This rather long discussion of the EAI 
has been given in order to make more or 
less clear, not only the basic operations 
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involved in its computation, but also 
certain of its implications. The EAI of 
a term, computed from data obtained 
under adequate conditions, is indicative 
of one of the most important character- 
istics of word usage, the relatively precise 
degree to which words may be regarded 
as factually meaningful—or vague. 

Use of the measure requires that it be 
computed from data obtained under 
known and specified conditions; more- 
over, the particular form of the basic 
formula to be used in computing it will 
vary somewhat with the nature and pur- 
pose of the investigation. It is proposed 
that preliminary work to be done in the 
present program will involve construc- 
tion of a test by means of which EAI’s 
for a number of different terms can be 
determined under a variety of condi- 
tions. Work already done indicates that 
the reliability of such a test can be ex- 
pected to be quite high, that its ad- 
ministration and scoring offer no insur- 
mountable problems, and that data ob- 
tained by means of it will reveal dif- 
ferences between words and between in- 
dividuals and groups. 

In the administration of this test it is 
planned that the subject will be given a 
word in a standard context, as in the 
statement: “Point to the pictures that 
show people doing good things.” The 
subject then points to such pictures, 
among a standard set of pictures, as to 
him represent referents of good as so 
used. Each picture in the set is num- 
bered and the number of each picture to 
which the subject points is recorded. 
From data so obtained from each of a 
group of subjects, the EAI of each word 
in the test is to be computed, as was done 
for the Gallup poll data presented in 
the preceding pages. 

As part of the present program, a study 
has been made by J. Wilson and the 
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writer (7) in which graduate students 
and instructors in psychology defined 
extensionally, by reference to a list of 
Statements taken from psychology texts, 
the terms “law,” “theory,” and “hypoth- 
esis." The mean EAI’s obtained were 
.62 for “law,” .40 for “theory,” and .28 
for “hypothesis.” 

Extensional Synonymity Index (ESI). 
Such EAI’s represent the relative degree 
of vagueness of words as used. By treat- 
ing the test data in other ways, they can 
be made to yield two other types of in- 
formation, represented by an extensional 


‘ synonymity index (ESI) and an exten- 


sional conformity index (ECI), respec- 
tively. By recording the percentage of all 
the subjects who point to each picture, 
or other types of referent, in defining 
each word, it is possible to measure the 
degree of synonymity between any two 


words. The formula ESI = 


bab. in 

V xy 
which c represents the percentage of 
subjects pointing to a given picture in 
defining both of two given words, and 
x and y represent the percentages of 
subjects pointing to the picture in de- 
fining each of the two words, respective- 
ly. This value is to be computed for 
each picture, and the values thus ob- 
tained for all the pictures are to be 
averaged in deriving an expression of 
the mean degree of synonymity between 
any given pair of words. 

Extensional Conformity Index (ECI). 
The percentages of subjects pointing to 
each picture in defining each word can 
also be used as word-fact relating be- 
havior norms. Thus, the pictures may 
be “weighted” according to these values, 
and on the basis of them the pointing or 
labeling of a given individual can be 
evaluated. For example, if a given indi- 
vidual in defining the word “good” 
were to point to certain: pictures, he 
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would be showing less conformity to the 
group that he would be in pointing to 
certain other pictures. The mean of the 
percentage values of the pictures to 
which an individual points in defining a 
given word would represent his degree 
of conformity to the group in his exten- 
sional use of that word. We may call this 
his extensional conformity index (ECI), 
and individual differences expressed in 
terms of the ECI might be found to be a 
factor in personality adjustment. 

The Intensional Agreement Index 
(IAI) expresses the degree of agreement 
among m” persons in defining a given 
term intensionally—i.e., by giving its 
verbal equivalents. A dictionary defini- 
tion is to be regarded as an intensional 
definition, as the term is here used. Like 
the EAI, the IAI may range in value 
from 0.0, representing no agreement, to 
1.0, representing maximum possible 
agreement. 

In a preliminary study carried out by 
N. Whitman and the writer (8), an at- 
tempt was made to determine IAI’s for 
each of certain terms used in the field 
of psychology (learning, perception, emo- 
tion, and personality) and certain terms 
used in the field of biochemistry (fats, 
lipids, enzymes, oxidation, and basal 
metabolism). Textbooks in each field 
were examined until for each term six 
definitions (from six different authors) 
had been found. These definitions were 
then edited so as to exclude all words 
except nouns, verbs, adjectives, and ad- 
verbs (the adverbs when and where, the 
adjectives that, these, those, and which, 
and articles used as adjectives were also 
excluded). Then for each term the num- 
ber of types (different words) used in all 
six definitions was recorded, and the 
number of definitions in which each 
type was used was determined. The num- 
ber of obtained agreements, in the use 


of any given type by the six textbook 
authors, was found by means of the for- 
mula (n — 1) .5n, in which n represents 
the number of definitions in which the 
type occurred. The values thus obtained 
for the various types were summed in 
determining the total number of ob- 
tained agreements shown by the six text- 
book writers in verbally defining the 
term in question. The maximum pos- 
sible number of agreements was com- 
puted by using the formula x(n — 1) .5n, 
in which n represents the total number of 
definitions, six in each case, and x repre- 
sents the total number of types used in 
all the definitions, The maximum pos- 
sible number of agreements was then di- 
vided into the obtainéd number of agree- 
ments in determining the IAI of a given 
term. The IAI’s as thus determined, 
were: 


Psychological terms No. of types 
Learning 024 44 
Perception .006 40 
Emotion .010 48 
Personality 007 46 

Ave. O12 44-5 

Biochemical terms 
Fats .o80 56 
Lipids 150 59 
Enzymes 127 20 
Oxidation .067 27 
Basal Metabolism 035 50 

Ave. 092 42.4 


The difference between the mean IAI's 
may be regarded as indicating a meas- 
urable difference between the fields of 
psychology and biochemistry with re- 
gard to the degree of terminological 
agreement that has been achieved in 
them to date. One important aspect of 
scientific development is to be observed 
in the fact that among biochemists at the 
present time there is a tendency to 
abandon the term fats in favor of the 
term lipids—a tendency to replace one 
term with another that has a higher 
IAI. Increasing agreement as to defini- 
tions, both intensional and extensional, 
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is a basic characteristic of the develop- 
ment of a science; and a means of 
measuring the degree of agreement that 
has been achieved within the various 
fields makes possible a peculiarly ob- 
jective comparison of them in this im- 
portant respect. Degree of similarity 
verbal formulations generally 
can be measured in terms of the IAI. 
The procedure followed in the above 
study of psychological and biochemical 
terms can be modified in at least three 
ways. First, the definitions can be ob- 
tained directly from the subjects rather 
than from text books or other published 
material. Second, the subjects can be 
instructed to define each word by listing 
synonyms of it, «and the number of 
synonyms to be listed can be limited. 
Third, the words to be defined need not 
be presented only in isolation, but they 
may be presented also in context, other 
words to be substituted by the subject 
for the word in question, or a definition 
to be written for the word as used in 
the particular context. The influence of 
differences in context on the meaning, 
and on agreement as to the meaning, of 
specific words can thus be investigated. 
Intensional Synonymity Index (ISI). 
From data of the type just discussed it 
is possible to obtain measures of inten- 
sional synonymity. Degree of synonymity 
of given pairs of words defined exten- 
sionally can be measured by means of 
procedures already described. Similar 
procedures can be used in the present 
connection. For example, suppose the 
words good and worthwhile to have been 
defined by each of 100 subjects, each 
of whom defined each word by listing 
three synonyms for it. The degree of 
intensional, or verbal, synonymity be- 
tween these two words can then be com- 


among 


mel OM 


puted by means of the formula 


xy 





in which c represents the number of 
terms (types) given by the 100 subjects 
as synonyms for both words, and x and 
y represent the number of terms (types) 
listed as synonyms for each of the two 
words, respectively. The correlation be- 
tween extensional and intensional syn- 
onymity indexes would be of interest. 

Semantic vocabulary test. As has been 
indicated previously in this outline, vo- 
cabulary measures can be obtained from 
a language sample obtained from any 
given individual in terms of type-token 
ratios, type frequencies, proportionate 
vocabulary, and standard frequency vo- 
cabulary. Another type of vocabulary 
test might be attempted. A common 
criticism of ordinary vocabulary tests is 
that while they are indicative of the 
number of words an individual “knows” 
or “recognizes,” they are not necessarily 
indicative of the range of “depth” of the 
individual’s knowledge of or skill in 
using each word that he “knows.” The 
problem raised by this criticism involves 
technical difficulties, but certain ap- 
proaches to its solution appear to be 
possible (14). 

Investigation could be made of the 
feasibility of constructing a vocabulary 
test of such a nature that the individ- 
ual’s ability to use each word would be 
sampled in detail. It is possible to distin- 
guish types of meaning, such as mean- 
ing in terms of use, variety, differentiat- 
ing characteristics, sources, etc. For ex- 
ample, the word orange can be defined 
in terms of (a) the various uses of 
oranges, (b) the kinds of oranges, (c) the 
characteristics that differentiate oranges 
from other things, (d) the geographical 
areas where oranges are grown, the meth- 
ods by which they are grown, the history 
of these methods, etc., and (e) in terms 
of the scientific research that has been 
done on oranges, the methods used in 
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picking, packing, processing, marketing, 
transporting, etc. This does not exhaust 
the problem of defining orange, but it 
illustrates the possibility of devising a 
vocabulary test of a type that should 
make possible a measure of vocabulary 
“depth” as well as “range.” 

Measures of “allness.” Previous men- 
tion has been made of “allness’”’ terms, 
such as all, everyone, nobody, every, 
never, absolutely, etc. Language spoken 
during moments of anger or despair, or 
other relatively profound affective states, 
appears to be particularly characterized 
by such terms. They give to language a 
character which reflects what is usually re- 
ferred to as dogmatism, or stubbornness, 
inflexibility, etc. Orientation on the basis 
of dichotomies, or of the excluded 
middle—a two-valued, either-or orienta- 
tion—appears to be basic to and to be 
fostered by, this sort of language. The 
degree to which one is prone to two- 
valued orientation is probably an im- 
portant aspect of one’s general adjust- 
ment, personality development, intelli- 
gence, etc. Insofar as it might prove 
possible to set up rigorous criteria of 
allness terms, the frequency of their use 
in language samples could be studied. 

Another approach to the study of all- 
ness, however, is also to be proposed. 
From one point of view allness may be 
regarded as manifested in extreme re- 
sponses in situations where they are 
not mandatory. An attempt could be 
made to construct a reliable test involv- 
ing, say, 100 items, to each of which a 
response can be made along a graduated 
scale expressive of extreme and _ inter- 
mediate degrees of preference, attitude, 
behaviorial tendency, etc. At least five 
and possibly seven or more alternative 
responses to each item should be pro- 
vided, one expressive of neutrality or 
average tendency and the others dis- 


tributed on either side and graduated 
toward the two extremes. The test would 
be scored, not in terms of the prefer- 
ences, etc. expressed, but in terms of the 
proportion of extreme (allness) responses. 
It is anticipated that two main types of 
evaluative tendencies might be indicated 
by such a test, the tendency to give 
extreme responses, or allness, and an ex- 
treme tendency to give indecisive, in- 
definite, neutral responses. The latter 
might characterize certain schizoid con- 
ditions, for example. It is to be noted 
that this type of test should get away 
from one common weakness of pencil- 
and-paper tests, in that the effect of 
falsified responses on the score will be 
minimized, since the “intensity” rather 
than the “content” of the responses will 
determine the score.‘ 

Tests of verbal differentiation. It 
would appear reasonable to assume that 
the adequacy of generalization or “ab- 
stract thinking” depends largely upon 
the adequacy of the analysis or dif- 
ferentiation upon which the generalizing 
is based, This is indicated by an exam- 
ination of practically any generalization 
process; it is especially obvious, perhaps, 
in medical diagnosis. The ability to ob- 
serve, respond to, and relate differences 
would appear to limit the ability to ab- 
stract similarities effectively. In fact, ab- 
stracting (roughly, generalizing) can be 
defined as a process of leaving out de- 
tails or differences; similarities are 
recognized and formulated in. accord- 
ance with the way differences are dis- 
regarded, not observed, or related. Con- 
sciousness of abstracting (9, 10), there- 
fore, in any given instance, is seen to 
depend on an awareness of the differ- 
ences that are being disregarded or re- 
lated in the abstracting of similarities. 


‘Previous work suggestive of this approach 
has been reported by Watson (19). 
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It is proposed to construct a test spe- 
cifically designed to measure an indi- 
vidual’s ability to express differences, or 
to perform verbal differentiation. It is 
the intention to begin with the simple 
procedural plan of presenting the sub- 
ject with pairs of objects, designs, etc. 
and requesting him to tell the differences 
between them. A time limit, to be deter- 
mined, is to be set for each response. An 
attempt is to be made to score the re- 
sponses in each of three ways. First, the 
mere length of response is to be meas- 
ured; it is hardly to be expected that this 
will suffice, except possibly as a very 
gross measure. Second, the number of 
differences enumerated is to be noted; 
it will be necessary to formulate rigorous 
criteria of a “difference.” Third, various 
forms of the type-token ratio are to be 
tried as possible expressions of the sub- 
ject’s level of performance. 

Assuming the construction of a re- 
liable test, scores on the test are to be 
related to other variables. The relation 
of differentiating ability to intelligence, 
as measured by current standard tests, 
and to other criteria of competence, is 
of particular interest. 


SUPPLEMENTARY MEASURES 


The entire research program here pro- 
posed involves not only the language be- 
havior but 
also certain other measures which are to 
be used in order to obtain data concern- 
ing the relation of the language measures 
to other aspects of behavior. Among 
these supplementary measures are tests 
of intelligence, measures of mental and 
chronological age, achievement and apti- 
tude tests, measures of silent and oral 
reading, of speech and writing, and vari- 


measures discussed above, 


ous indices of personality. 
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STUDIES COMPLETED 

To date six studies have been com- 
pleted, and a considerable amount of 
preliminary or exploratory investigation 
has been done. The six completed studies 
have been done by Fairbanks (5), Mann 
(11), Chotlos (4), Tuthill (18), Johnson 
and Whitman (8), and Johnson and Wil- 
son (7). The investigations so far com- 
pleted have been concerned mainly with 
problems of method, although they have 
been designed to contribute, also, to a 


fuller understanding of language _be- 
havior in its various relationships. 
Individual 3,000-word spoken _lan- 


guage samples were obtained by Fair- 
banks (5) from each of 10 schizophrenic 
patients and 10 university freshmen. 
Mann (11) obtained 2,800-word written 
language samples from each of 24 schizo- 
phrenic patients and 24 university fresh- 
men. The writer has obtained 3,000- 
word written samples from each of ap- 
proximately 1,000 Iowa public school 
children, selected on the basis of age, 
sex, 1.Q., type of school (rural, town, 
city) and socio-economic status.® A se- 
lected set of 108 of these written lan- 
guage samples have been analyzed in con- 
siderable detail by Chotlos (4). 

The studies which follow the present 
article in this monograph will serve to 
illustrate some of the above types of 
approach to the investigation of lan- 
guage behavior. ‘The present discussion 
is offered as a general introduction to 
these and to the further studies that will, 
it is hoped, be included in the program 
of research which has here been outlined. 

* Acknowledgement is hereby made of grants 
from the Work Projects Administration for lowa 
Projects 4,892 and 5,960, by means of which 
these data were obtained and tabulated, and to 
Professor George D. Stoddard, then Director of 


the Iowa Child Welfare Research Station, who 
sponsored the projects. 
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I. INTRODUCTION 


B Bie GENERAL PROGRAM Of research, of 
which this study is a part, has been 
outlined in a previous article by John- 
son (g). As the first study to be under- 
taken within that program, this investi- 
gation is concerned primarily with prob- 
lems of method. It is concerned specific- 
ally with a partial exploration of the 
possibilities of measuring certain aspects 
of language behavior, and of differentiat- 
ing samples of spoken language in terms 
of the measures employed.? It is assumed 
that the first step in a comprehensive 
language behavior research program lies 
in the attempt to develop adequate tech- 
niques of measurement necessary for the 
formulation and testing of hypotheses. 
In accordance with this point of view, 
an attempt was made in this study to 
obtain two groups of language samples 
which might be assumed to be sufficiently 
different as to make possible some indi- 
cation of the sensitivity of the measures 
to be employed. On the basis of this 
‘This study was done in the Department of 
Phychology at the State University of Iowa as a 
dissertation in partial fulfillment of the require- 
ments for the degree of Doctor of Philosophy. 
It is part of a program of research on language 
behavior and was directed by Wendell Johnson. 
The writer is grateful to Dr. Andrew H. Woods, 
Director, and the staff of the Iowa State Psycho- 
pathic Hospital [1938-40]; and to Dr. Leonard P. 
Ristine, Superintendent, and the staff of the Mt. 
Pleasant State Hospital, for their cooperation 
in securing subjects for the investigation. Special 
acknowledgment is made of the assistance of Dr. 
Frank Robinson, resident psychiatrist at the 
lowa State Psychopathic Hospital during 1938-39. 
7A companion study by Mann [11] is con- 
cerned with written language. The specific meas- 
ures used in that study and this one, as well as 


several other types of language measures, are 
discussed in the above mentioned article by John- 


son [9]. 
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consideration it was decided to obtain 
the language samples from hospital pa- 
tients suffering from schizophrenia, and 
from university freshmen who were 
judged to be superior in terms of criteria 
to be indicated in the section on Pro- 
cedure. The ‘superior’ freshmen were 
chosen with the expectation that they 
would furnish relatively ‘adequate’ lan- 
guage, and the schizophrenic patients 
were used on the assumption that their 
language would prove to be relatively 
‘inadequate’, and that the contrast might 
be sufhciently marked to be quantita- 
tively expressed. One of the important 
clinical manifestations of schizophrenia 
is to be noted in the language of the 
patients suffering from the disease (19). 
As the illness progresses there is a tend- 
ency for the language to appear discon- 
nected, illogical, even incomprehensible. 
Stereotypy in verbal expression is fre- 
quently apparent. Thus, there would 
seem to be reasonable ground for ex- 
pecting that the language of schizo- 
phrenic patients might be demonstrably 
different, quantitatively, from that of 
‘superior’ normal subjects. Relevant 
studies have been reported by White 
(15), Woods (16), and Cameron (2, 3). 
It was an incidental consideration that 
any differences that might be revealed 
as between these two groups would pos- 
sibly be of psychiatric and psychological 
interest. The primary purpose of the 
investigation is, however, methodologi- 
cal, and any conclusions to be drawn 
from the findings with regard to the 
nature of ‘schizophrenic language’ are 
to be most carefully evaluated. In this 
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connection, it is to be emphasized that it 
was regarded as of first importance to 
obtain two groups of subjects who might 
be expected with some assurance to pro- 
duce demonstrably different language 
samples. It was for this reason that ‘supe- 
rior’ university freshmen were selected. 
This meant, however, that the possibil- 
ity of securing schizophrenic patients 
matched with the freshmen in terms of 
intelligence and educational background 
—a difficult undertaking in any case— 
was deliberately jeopardized. Most of 
the patients were judged to be of aver- 
age intelligence or above, as will be in- 
dicated later, but the fact remains that 
any demonstrated language differences 
between the patients and the students 
may be due, in part, to differences in 
intelligence or in scholastic background, 
and not entirely to schizophrenia, per se. 
It appeared advisable, nevertheless, to 
establish first, as far as possible, the de- 
gree to which the measures used were 
sensitive or differentiating. Had well 
matched groups been used and no differ- 
ences in language found, the basic ques- 
tion of the differentiating value of the 
measures would have remained unan- 
swered. It could not have been con- 
cluded whether there were no differences 
to be measured, or that the measures 
used were too crude to reveal them. 
Therefore, the methodological problem 
was placed first in importance in design- 
ing the study, but schizophrenic patients 
were used in the hope that, if the meas- 
ures turned out to be differentiating, 
some findings of psychiatric and _psy- 
chological significance might be gained. 


II. PROCEDURE 


Two groups of adults served as sub- 
jects in this study: (1) ten psychotic pa- 
tients diagnosed as schizophrenic; (2) ten 
freshmen at the State University of Iowa. 


The major characteristics of these groups 
are summarized below. 

Of the schizophrenic subjects, four 
were patients at the Iowa State Psycho- 
pathic Hospital, Iowa City, and the 
other six, three of whom had previously 
been in the Iowa State Psychopathic 
Hospital, were committed patients at 
the Mt. Pleasant State Hospital at Mt. 
Pleasant, Iowa. These ten patients were 
chosen on the basis of the certainty of 
the diagnosis made of them by the psy- 
chiatrists and the possibility of securing 
their co-operation in the proposed inter- 
view situation. Data concerning the in- 
dividual cases are as follows: 

Case 1. Diagnosis: schizophrenia, para- 
noid type. A single male, aged 46 years, 
6 months; educated through gth grade 
and one year business college; first psy- 
chotic episode in 1916, confined to the 
Mt. Pleasant State Hospital continuously 
since 1934; scored Intelligence Quotient 
of 114 on Wechsler-Bellevue Adult Test 
and 104 on Revised Stanford-Binet Test, 
Form L, passing the vocabulary test on 
the latter at the Superior Adult III 
level; patient inclined to give up easily 
on tests; psychometrist commented: “in- 
tellectual development has been supe- 
rior.” 

Case 2. Diagnosis: schizophrenia, hebe- 
phrenic type. A single male, aged 31 
years; educated through 8th grade; first 
mental symptoms in 1935, confined in 
Mt. Pleasant State Hospital since; scored 
Intelligence Quotient of 24 on Revised 
Stanford-Binet Test, Form L, but so de- 
teriorated psychometrist felt no estimate 
of original intellectual level possible. 

Case 3. Diagnosis: schizophrenia, cata- 
tonic type. A single male, aged 20 years, 
8 months; educated through 11th grade; 
first mental symptoms in 1939, dis- 
charged from Iowa State Psychopathic 
Hospital several months later as much 
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improved; scored Intelligence Quotient 
of 62 on Stanford Revision of Binet- 
Simon Test but excited and distractible; 
six weeks later when more co-operative 
scored Intelligence Quotient of 99 on 
retest and 115 on Revised Stanford- 
Binet Test, Form L, passing the vocab- 
ulary test on the latter at the Superior 
Adult II level; original level judged by 
psychometrist to have been “high aver- 
age to superior.” 

Case 4. Diagnosis: schizophrenia, para- 
noid type. A widower, aged 41 years, 9 
months; educated through 8th grade; 
first mental symptoms in 1934, confined 
in Mt. Pleasant State Hospital since 
1938; had an Intelligence Quotient of 76 
on Revised Stanford-Binet Test, Form L, 
scoring slightly below average on vocab- 
ulary test; required considerable urging 
before trying tests; original level esti- 
mated by psychometrist to have been 
“slighty below average.” 

Case 5. Diagnosis: schizophrenia, para- 
noid type. A single male, aged 31 years, 
four months; educated through high 
school and business college; expressed 
paranoid ideas in 1930 and 1935 and 
developed acute symptoms in 1939, dis- 
charged after several months from Iowa 
State Psychopathic Hospital as im- 
proved; scored Intelligence Quotient of 
87 on Stanford Revision of Binet-Simon 
Test, passing vocabulary test at high 
average level; passed vocabulary test on 
Revised Stanford-Binet, Form L, at 
Superior Adult I level; original level felt 
by psychometrist to have been ‘average 
intelligence or above.” 

Case 6. Diagnosis: schizophrenia, hebe- 
phrenic type. A single female, aged 36 
years, 11 months; educated through high 
school and two years’ college; first mental 
symptoms in 1932, hospitalized at Iowa 
State Psychopathic Hospital in 1933, 
then committed to Mt. Pleasant State 


Hospital and there since; scored Intel- 
ligence Quotient of 83 on Revised Stan- 
ford-Binet, Form L, passing vocabulary 
test at Superior Adult II level; judged 
by psychometrist to have been originally 
“at least high average.” 

Case 7. Diagnosis: schizophrenia, para- 
noid type. A single female, aged 27 years, 
1 month; educated through two years’ 
college; first mental symptoms in 1930, 
present episode began in 1938, hospital- 
ized at Iowa State Psychopathic Hospital 
in 1939, then committed to Mt. Pleasant 
State Hospital and there since; scored 
Intelligence Quotient of 118 on Stanford 
Revision of Binet-Simon Test, passing 
vocabulary test at very superior level, 
and 138 on Revised Stanford-Binet, 
Form L, passing vocabulary test at 
Superior Adult III level; psychometrist 
commented that intellectual level was 
“very superior.” 

Case 8. Diagnosis: schizophrenia, un- 
classified type. A single female, aged 37 
years, 1 month; educated through pre- 
paratory school and four years’ college, 
in Biblical seminary at time of first men- 
tal symptoms in 1934; hospitalized at 
Iowa State Psychopathic Hospital, 1939, 
discharged home after several months as 
much improved but symptoms gradually 
returning. 

Case 9. Diagnosis: schizophrenia, para- 
noid type. A married female, aged 45 
years, 3 months; educated through 8th 
grade; first mental symptoms in 1937, 
hospitalized at Iowa State Psychopathic 
Hospital, 1939, and discharged several 
months later as unimproved with advice 
to commit patient to a state hospital. 

Case ro. Diagnosis: schizophrenia, par- 
anoid type. A married female, aged 31 
years, 3 months; educated through high 
school and business school; first mental 
symptoms in 1939, hospitalized at Iowa 
State Psychopathic Hospital for several 
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weeks, then transferred to Independence 
State Hospital, Independence, Iowa, and 
there since; scored Intelligence Quotient 
of g2 on Stanford Revision of Simon- 
Binet Test, passing vocabulary test at 
superior level; psychometrist com- 
mented: “vocabulary and the quality of 
her responses indicate superior intel- 
ligence”’, 

In summary, the schizophrenic sub- 
jects consisted of five males and five fe- 
males, ranging in age from 20 years, 8 
months, to 46 years, 6 months; six had 
been diagnosed as paranoid, two as hebe- 
phrenic, one as catatonic, and one had 
not been classified. The length of the 
illness ranged from an acute episode 
lasting about a month to an illness that 
began in 1916 and has gradually shown 
exacerbations since. The educational 
backgrounds ranged from 8th grade to 
college graduation; one patient was felt 
by the psychometrist to be of very supe- 
rior intelligence, two superior, two high 
average to superior, one average or 
above, one slightly below average, and 
one too deteriorated to permit evalua- 
tion of original level. It was not possible 
to obtain psychometric ratings on the re- 
maining two patients; of these, one grad- 
uated from college and one had no 
training beyond the 8th grade but was 
considered an excellent business man- 
ager by a local attorney. 

The freshman students who formed 
the second group were chosen on the 
basis of their September, 1938, scores on 
the Iowa Qualifying and Placement Ex- 
aminations. All ranked from the gist to 
the ggth percentile on Silent Reading, 
Comprehension, and from the g5th to 
the ggth percentile in English Training. 
It can be assumed that the intellectual 
level of these subjects is probably supe- 
rior, as a recent unpublished study by 
Mitchell (12) indicated a correlation of 
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.76 between the Intelligence Quotients 
of 66 freshmen as scored on the Revised 
Stanford-Binet, Form L, and their Com- 
posite Score on the Iowa Qualifying and 
Placement Examination, the average In- 
telligence Quotient being 122. The 
group of ten freshmen was chosen on 
the assumption that its members would 
represent relatively adequate language 
usage. Six of the freshmen were female 
and four were male; the age range was 
from 17 years, 5 months, to 19 years, 1 
month. They came from homes in which 
the following occupations were repre- 
sented by the wage earners: bank re- 
ceiver, jeweler, theatre owner, coal 
miner, postmaster, county superintend- 
ent of schools, life insurance agent, lum- 
berman, odd jobs and trucking, indus- 
trial engineer and sales manager. 

A consideration of the methods to be 
used in treating the data and of the 
issues with which the study was involved 
seemed to indicate that a 3,000-word 
spoken language sample from each sub- 
ject would be adequate. In formulating 
the procedure care was taken to secure 
samples that would be comparable from 
subject to subject and group to group. 
Because of the frequent difficulty found 
in getting schizophrenic patients to talk 
readily, the following interview situa- 
tion was prepared, utilizing 14 proverbs 
whose efficacy as stimuli has been demon- 
strated in previous studies done at the 
Iowa State Psychopathic Hospital. The 
following instructions were given to each 
subject: 

“T want you to talk about some prov- 
erbs today. You know what a proverb 
is. A proverb is a sentence that teaches a 
lesson. I am going to read some proverbs 
to you, and I want you to tell me what 
they mean. I also want you to describe a 
situation in which each proverb would 
apply. For example, the proverb ‘Let 
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sleeping dogs lie’ means that we should 
avoid stirring up old troubles or quar- 
rels. An example of a situation in which 
this proverb would apply would be, for 
instance, if you and a friend had quar- 
reled over something several months ago, 
you should forget it and be friends with 
him again instead of continuing to quar- 
rel with him each time that you see him. 
Do you understand what I mean? Now 
you tell me what this proverb means, 
‘The early bird catches the worm.’ 

“Now give me an example illustrating 
that.” 

This procedure was continued with 
each of the following proverbs: 

“He who laughs last laughs best.” 

“A chain is as strong as its weakest 
link.” 

“The devil finds work for idle hands.”’ 

“Tell me the company you keep and 
I'll tell you what you are.” 

“Deeds are males and words are fe- 
males.” 

“Like father, like son.” 

“What you sow you will reap.” 

“Barking dogs never bite.”’ 

“You can’t touch pitch without being 
tarred.” 

“A crow is known by the company he 
keeps.” 

“A fair face may hide a foul heart.” 

“A prophet is without honor in his 
own country.” 

“It is always darkest just before the 
dawn.” 

The subjects were asked to continue 
talking about anything that they wished 
to after finishing the proverbs. It was 
difficult to keep the interview situation 
as simple for the schizophrenics as for 
the freshmen, as would be expected with 
psychotic individuals who show so little 
response to their environment, and it 
was necessary to stimulate them more 
frequently with such questions as why 


they were in the hospital and what they 
were doing, in order to get the requisite 
3,000 words from each. Two patients had 
to be interviewed a second time in order 
to get enough words, the second inter- 
view continuing where the first had left 
off. In one of these cases the total num- 
ber of words still did not approximate 
3,000, and as the patient was removed 
from the hospital by relatives before a 
third interview could be arranged, his 
language sample consists of only 2,800 
words. The patients were interviewed by 
a resident psychiatrist at the Iowa State 
Psychopathic Hospital, while the experi- 
menter interviewed the freshman sub- 
jects. All interviews were completely re- 
corded by means of an electrical dicta- 
phone apparatus, consisting of a micro- 
phone, amplifier, and two dictaphones. 
All recordings were continuous. As the 
microphone was concealed among books 
and papers on the interview desk, the 
subjects were not aware of the fact that 
their speech was being recorded except 
in the case of one freshman who hap- 
pened to uncover the microphone. How- 
ever, it was the opinion of the inter- 
viewer that even in this case speech was 
not disturbed. 

The dictaphone records were then 
transcribed by the experimenter, follow- 
ing the conventional forms of word divi- 
sion and spelling as closely as possible. 
The neologisms or coined words occa- 
sionally introduced by the schizophrenics 
were spelled as they sounded phonet- 
ically. As would be expected, the intel- 
ligibility of the records varied in accord- 
ance with the amount of intensity and 
the clearness of articulation used by the 
various subjects. Each record was played 
over until the experimenter was reason- 
ably sure of the transcription. All words 
and sections which were doubtful were 
omitted. 
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A study by Betts (1) has indicated that 
fewer than one per cent of the words of 
normal speakers recorded by the elec- 
trical dictaphone technique are unin- 
telligible. However, the percentage of 
such words is probably higher in the 
present study due to occasional mum- 
bling by the patients, but it cannot be 
stated definitely just how much. As 
stated before, the experimenter played 
the records over until reasonably certain 
of the transcription, omitting all words 
or phrases that were doubtful. 

The language sample of each subject 
was divided into 30 consecutive segments 
consisting of 100 words each. A word 
count was then made for each protocol 
by placing a tally mark for each different 
word on tabulation sheets so organized 
that each 100-word segment could be 
tabulated individually. The part of 
speech for each word was designated as 
it was tabulated. The following rules 
were followed in determining what con- 
stituted a word: 

1. Each group of letters separated by 
spaces on both sides from adjacent 
groups of letters was counted as a word, 
even though it might be part of a place 
name, as in Des Moines (two words), an 
initial, as in John D. Rockefeller, Jr. 
(four words), and abbreviation of a word 
previously used, as coop. for cooperative, 
a spelling of a word previously pro- 
nounced, as p-a-r-d for pard (one word), 
or a neologism coined by a schizophrenic 
patient, as tombody. 

2. Random letters given consecutively 
by schizophrenic patients, such as d-t, 
were considered as spellings and counted 
as one word. 

3. Any number was counted as one 
word; for example, 125 was tabulated as 
one word. 

4. A hyphenated word was counted as 
one word, Webster’s New International 
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Unabridged Dictionary (14) being used 
as the authority as to whether or not a 
word should be hyphenated. 

5. Sounds like uh and er uttered by 
subjects during pauses were not consid- 
ered as words. However, in one case uh 
and er were cited by a subject as exam- 
ples, in which instance they were re- 
garded as words. The sounds huh, uh 
huh, and hunh uh were also regarded as 
words, being tabulated under what, yes, 
and no respectively. 

6. Each time a word was used as a 
different part of speech it was counted as 
a different word. For example, mine as 
a noun and mine as a pronoun were 
tabulated as two different words. 

4. Different tenses of a verb having 
identical spellings were counted as dif- 
ferent words.-For example, read, present 
tense, and read, past tense, were tabu- 
lated as two different words. 

8. Common nouns and proper nouns 
having identical spellings were thrown 
together. For example, the two words, 
Death Valley, were tabulated under the 
common nouns, death and valley. 

The data taken from these tabulation 
sheets were organized into three different 
sections of results: (1) the type-token 
ratios, (2) grammatical analysis, (3) word 
frequencies (8, 9). 


Ill. RESULTS 


1. Type-token ratio. This measure is 
computed by dividing the number of 
different words (types) by the total num- 
ber of running words (tokens). Since the 
number of different words decreases as 
successive increments are added to a lan- 
guage sample (4), the number of tokens 
used in computing the type-token ratio 
must be kept constant in order to deter- 
mine any variations within any given 
language sample, or in order to make 
the ratio comparable from one sample 
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to another. In this study 100 was used as 
the standard number of tokens, each 
language sample having been divided up 
into 30 consecutive 100-word segments. 
The TTR for each of these 100-word 
segments was then computed. 

To determine, first, the internal con- 
sistency (i.e., how well a random half of 
the sample measures what the whole 
sample measures) of the 3,000-word sam- 
ple for each subject, the t-test for related 
measures (10) was used. This was com- 
puted by dividing at random the go 
TTR’s® for each subject into two sets 
and finding the group mean for each 
half. From this procedure there resulted 
two sets of ten means each for each group 
of subjects. Each set of ten means was 
averaged, giving two mean values for 
each group of subjects. The difference 
between these two mean values was eval- 
uated. The value of ¢ for the difference 
between the two means thus obtained 
for the schizophrenic patients was .219, 
and that for the freshmen was .430. As 
neither of these values of t, with nine 
degrees of freedom, is significant at the 
5 per cent level of confidence it would 
appear that there is no reliable differ- 
ence between the two means for each 
group, or that the internal consistency 
of the language samples is high. 

A test of the hypothesis that there is 
no difference between the variances of 
the distributions of the SD’s of individ- 
ual samples of the schizophrenic patients 
and of the freshmen is afforded by the F 
test (10). It will be recalled that each 
individual sample is made up of go seg- 
ments, for each of which a TTR was 
computed. When F was computed as the 


ratio of the variance of the distribution 


* As the language sample of one schizophrenic 
patient consisted of only 2,800 words, because he 
was withdrawn from the hospital before 3,000 
words could be obtained, only 28 TT'R’s were 
obtained in his case. 


of the SD’s for the schizophrenic patients 
to that for the freshmen, the value ob- 
tained was 2.2. Since the value of F, with 
nine and nine degrees of freedom, 
needed for significance at the 5 per cent 
point is 3.18, the hypothesis of no sig- 
nificant difference is tenable. That is to 
say, the I°T'R’s of the schizophrenic pa- 
tients did not vary more from segment 
to segment than did those of the fresh- 
men. 

Table 1 gives the distribution of the 
mean segmental IT’T'R’s for the individ- 
ual freshmen and schizophrenics; each 
individual mean represents the average 
of the go segmental TTR’s computed 
for each sample. This table indicates a 
tendency for the mean TTR to be gen- 
erally lower in the case of the schizo- 
phrenics, only one freshman having a 
lower ratio than the patients with the 
highest ratios. It is to be noted, also, that 
the range for the schizophrenic group is 
much greater than for the freshman 
group, extending from .49 to .62 for the 
former, and from .61 to .67 for the latter. 

The group mean TTR for the schizo- 
phrenics was .57, with a standard error 
of .o124, and that for the freshmen was 
.64, with a standard error of .0043. In 
order to test the significance of the dif 
ference between these two means the f- 
test (10) was applied. The value obtained 
for t was 5.61, which, with 18 degrees of 
freedom, is significant at the 1 per cent 
level of confidence. Therefore, the hy- 
pothesis that these two samples were 
drawn from populations whose means 
are equal may be rejected. 

However, one of the assumptions un- 
derlying the t-test when used to test the 
significance of the difference between 
means of independent small samples is 
that the true variance of the population 
from which one sample is drawn must 
be equal (or approximately equal) to the 
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TABLE 1 


Mean TTR’s for the individual subjects ranked in descending order 





Schizophrenic patients 


Freshman subjects 





Mean TTR S.D. C.V. 


.62 .048 7.74 
.O1 -044 7.21 
.60 .048 8.00 
.58 .O50 8.62 
-57 .O71 12.46 
.56 .030 8.93 
-50 .050 10.00 
~55 .004 11.64 
-53 -O71 13.40 


-49 .0606 13.47 


Mean TTR S.D. 





RY. 
.67 .056 8. 36 
.66 .037 5.01 
. 66 .035 5-30 
.64 .040 6.25 
.64 .057 8.Q1 
.64 .057 8.91 
.64 .053 8.28 
.63 .053 8.41 
.63 .O42 6.67 
.OI -057 9.34 





true variance of the population from 
which the other sample is drawn. In 
order to discover whether or not this 
assumption is valid in these samples the 
F test was applied. When F was com- 
puted as the ratio of the variance of the 
distribution of the mean T’TR’s for the 
schizophrenics to that of the freshmen, 
the value obtained was 8.36, which, with 
nine and nine degrees of freedom, is 
significant at the 1 per cent point. It 
might be possible to interpret this as in- 
validating the above use of the t-test with 
these data. There is doubt on this point, 
and while some statisticians might ac- 
cept the t-test as here applied, it was 
thought best to treat the data in another 
and somewhat different way. Conse- 
quently, as a further check on the re- 
liability of the difference between the 
two group means, t was used to set limit- 
ing values for each group outside of 
which any exact hypothesis as to the 
value of the true mean may be rejected 
with a given degree of confidence (10). 
At the 1 per cent level of confidence the 
limiting values of the true mean for the 





patients were .6085-5277, and for the 
freshmen they were .6556-.6276. Since 
there is no overlap in these confidence 
intervals, we may be practically certain 
that the difference between the group 
mean ‘I’TR for the schizophrenics and 
that for the freshmen indicates a real 
difference between the two groups. 

In general it may be concluded that 
the schizophrenic patients tended to 
have lower mean segmental TTR’s than 
did the freshmen. In other words, the 
schizophrenic patients employed smaller 
vocabularies than did the freshmen. 

Interpretation of these differences in 
regard to the I’T'R’s of the schizophrenic 
patients and the freshmen must neces- 
sarily be made with caution because of 
several variables in the two groups, espe- 
cially within the schizophrenic group, 
such as age, time of onset of illness, in- 
tellectual level and educational advan- 
tages, which the experimenter was not 
able to control rigidly within the limita- 
tions of this study. However, two pos- 
sible relationships may be pointed out, 
namely, that existing between the intel- 
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lectual level and the TTR and that ex- 
isting between certain clinical pictures 
presented by the patients and the TTR. 

From a preliminary study by Zipf (17) 
in which he used a measure similar in 
some respects to the I’TR, it may be in- 
ferred, although it cannot be stated con- 
clusively, that the IYTR probably cor- 
relates positively with mental age. When 
the schizophrenic patients are ranked ac- 
cording to their mean TTR and what 
estimates could be obtained of their 
original intellectual level, it would ap- 
pear that a positive correlation would 
result, 


Schizophrenics 


Case Type 

7 paranoid 
3 catatonic 
9 paranoid 
1 paranoid 

10 paranoid 
8 unclassified 
6 hebephrenic 
5 paranoid 
{ paranoid 
2 hebephrenic 


Certainly the highest mean TTR was 
made by the schizophrenic with the 
highest intelligence, while the three low- 
est I°T'R’s were made by the three pa- 
tients with probably the lowest intel- 
ligence. Five other patients with prob- 
ably high average to superior intelli- 
gence ranked in between. 

No statements characterizing the vari- 
ous types of schizophrenia in terms of 
the T’TR would be justified by the above 
tabulation. 

Despite the probability that a positive 
correlation exists between the TTR and 
the intellectual level, the fact still re- 
mains that there were differences be- 
tween the mean T’TR’s for the schizo- 
phrenics who ranked highest intellec- 
tually and most of the freshmen. As the 
TTR represents the relationship —be- 


Mean TTR 


62 
61 
60 
58 
‘57 
56 
56 


dled 


OO 
53 
49 


tween the number of different words and 
the total number of words, the lower 
TTR’s of the schizophrenics would ob- 
viously indicate a smaller number of dif- 
ferent words used, hence more repeti- 
tions of the same words. Clinically, 
schizophrenic patients present a_tend- 
ency to repetition of behavior known as 
stereotypy which may be of attitude, 
movement, or speech. When the same 
word, phrase or sentence is repeated the 
stereotypy is known as_ verbigeration 
(13). It is possible, then, that the lower 
mean TTR’s for the patients represent 
to some degree in a quantitative manner 


Estimates of Intelligence or Education 


“Very superior” 

“High ave. to sup.” 

Eighth grade edu. 

“Superior” 

“Superior” 

College grad. 

“At least high ave.” 

“Ave. or above” 

“Slightly below ave.” 

“Too deteriorated to estimate.” 
Eighth grade education 


this clinical picture of stereotypy. 

2. Gramatical analysis. For this part 
of the study eight conventional parts of 
speech were used, namely, nouns, pro- 
nouns, verbs, adjectives, adverbs, prepo- 
sitions, conjunctions and _ interjections. 
The articles were tabulated separately 
and then considered both alone and in 
conjunction with the adjectives. For the 
classification of words on this basis, the 
following rules were followed: 

1. A noun used as an adjective was 
tabulated as an adjective only if the 
dictionary (14) gave the adjectival use 
as possible. For example, family in the 
combination family prayers was consid- 
ered as an adjective as the dictionary 
gives this usage. However, the word foot- 
ball in the combination football cham- 
pionship was tabulated as a noun as ne 
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TABLE 2 
Relative frequency of usage of the different parts of speech expressed as percentage of the total number 
of words used by the two groups, 29,800 in the case of the schizophrenic patients, and 30,000 
in the case of the freshman subjects. The range values are from the individual samples 





Schizophrenic patients 





Freshman subjects 

















% Range % Range 

Nouns 13.04 10.40-16.63 15.39 12.67-18.53 
Pronouns 22.68 19 .33-24.73 17.96 I14.40-20.40 
Verbs 26.28 24.27-30.47 22.95 20.50-24.47 
Adverbs 11.54 7.00-17.97 10.16 8.87-11.20 
Conjunctions 6.53 4.10- 8.77 8.83 7-33-11.40 
Prepositions 7.48 4.30-10.00 10.00 8 .80-11.00 
Interjections 2.64 -53- 4-43 1.26 .47- 2.00 
Adjectives 5-37 3-77- 7-10 6.69 5.67— 7.87 
Articles 4.48 2.53- 6.87 6.79 5.27- 9.07 
Adjs. and Arts. 9.85 8.60-12 I 


40 13.48 11.43-10.40 





adjectival use is mentioned in the dic- 
tionary. 

2. Participles were classes as adjectives 
and gerunds as nouns only when this was 
indicated as permissible by the diction- 
ary. Otherwise, they were classed with 
the verbs. 

3. All pronouns were classified under 
pronouns whether modifying nouns or 
not, 


4. The neologisms or coined words of ° 


the patients were interpreted according 


TABLE 3 
Values of ¢ and F obtained from testing signifi- 
cance of the difference in usage of certain 
grammatical categories, based on percentages of 
total sample, between schizophrenic patients and 
freshmen 





Values of t Values of F 





Adjectives 3.23 2.61 
Adverbs 1.44 10.10 
Nouns 2.50 1.53 
Pronouns 5-30 1.42 
Verbs 3.92 2.19 
Adjs. and Articles 5-34 1.58 
Articles 4.20 I.12 
Prepositions 5.04 4.50 
Conjunctions 3.43 1.44 
Interjections 2.98 6.72 





With 18 degrees of freedom, the values of ¢ 
required for significance are: at the 1% level of 
confidence t= 2.88; at the 5% level of confi- 
dence t= 2.10. 

With 9 and 9 degrees of freedom, the values of 
F required for significance are: at the 1% point 
F=5.35; at the 5% point F=3.18. 


to the parts of speech that they seemed 
functionally to assume in the sentence; 
if in isolation, they were considered as 
nouns. 

‘Table 2 gives the results of this gram- 
matical analysis for the schizophrenics 
and freshmen, respectively. ‘The t-test 
was applied to test the significance of the 
differences between the various percent- 
ages for the two groups, and the values 
of t obtained are given in Table 3. As 
can be seen, all of the ¢ values thus ob- 
tained are significant at the 1 per cent 
level, except that for nouns which is sig- 
nificant at the 5 per cent level, and that 
for adverbs, which is not significant at 
that level. From this we may conclude 
that the schizophrenic patients used sig- 
nificantly fewer nouns, conjunctions, 
prepositions, adjectives, and articles than 
did the freshmen, and significantly more 
pronouns, verbs, and interjections. 

The F ratio, involving the variances 
of the distributions of percentages (based 
on total words per sample) for each 
grammatical category for the two groups, 
resulted in the values of F also given in 
Table 3. Here the only significant results 
were with respect to the adverbs and in- 
terjections, which were significant at the 
1 per cent point, and the prepositions 
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which was significant at the 5 per cent 
point. This would indicate that only in 
the use of adverbs, prepositions, and in- 
terjections did the schizophrenic patients 
show significantly greater variability 
than did the freshmen. 

The ranges shown in Table 2 repre- 
sent the highest and lowest percentage 
for each part of speech in the individual 
language sample for each subject, the 


TABLE 4 


Relative frequency of use of the different parts 
of speech expressed as percentage of the total 
number of words used by the two groups, 29,800 
for the schizophrenic patients and 30,000 for the 
freshman subjects, compared with data from 
French, Carter, and Koenig (6) on telephone 
conversation. 











seri Schizo- _ Fresh- 
pray phrenics men 
Nouns 15.91 12.04 15.39 
Pronouns 18.22 22.68 17.96 
Verbs 22.39 26.28 22.95 


Adjs. and advs. 10.06 16.91 16.85 
Preps.andconjs. 12.62 14.01 18.83 
Articles 5.60 4.48 6.79 
interjections 8.08 2.64 1.26 





total number of words being 3,000 in 
each instance, except for the one patient 
who had only 2,800. It will be noted that 
the schizophrenic patients showed a 
greater range for all parts of speech ex- 
cept the pronouns and the adjectives and 
articles combined, where the freshmen 
had a slightly greater range. 

Table 4 shows the group percentages 
for each part of speech for the schizo- 
phrenics and for the freshmen, as com- 
pared with percentages computed from 
data given by French, Carter, and 
Koenig (6), in a study of telephone con- 
versations. The data taken from this 
study were reorganized, wherever given 
in such form as to make it possible, in 
order to make them more nearly com- 
parable to those of the present study. 
However, there were some differences in 


the French, Carter, and Koenig material 
that could not be changed so as to make 
it accord with that of the present study. 
For example, they classified all forms of 
yes and no under interjections, while 
such words were classed as adverbs in 
the present study, and they also classified 
laughter as an interjection, while it was 
ignored in this study. In addition, they 
grouped letters and numbers together 
under a separate heading, not classifying 
them as representing a part of speech, 
while letters were usually called nouns 
and cardinal numbers, adjectives, in this 
study. Therefore, this group of items, 
representing 5.05 per cent of the total 
number of words in their study, was ig- 
nored in the comparisons. These differ- 
ences in procedure explain to some ex- 
tent why the percentages of adjectives 
and of adverbs in the French, Carter, 
and Koenig data are considerably smaller 
than those for either of the two groups 
considered by the present experimenter, 
and why the percentage of interjections 
is considerably larger. However, it is in- 
teresting to note that the percentages for 
nouns, pronouns, and verbs in the 
French, Carter, and Koenig study ap- 
proximate very closely the corresponding 
percentages for the freshman group used 
in this study, and hence are lower for 
pronouns and verbs than are those of 
the schizophrenic group, while the per- 
centage of nouns is higher. In regard to 
prepositions and conjunctions _ the 
French, Carter, and Koenig percentage 
is lower than that for both the schizo- 
phrenic and freshman groups, but it 
more closely approximates that for the 
schizophrenic group. The percentage of 
articles in telephone conversation lies 
almost exactly half way between the per- 
centage of articles for the schizophrenics 
and that for the freshmen. 

Table 5 presents data from Horn (7) 
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TABLE 5 
Relative frequency of use of the different parts of speech expressed as percentage of the total number 
of words used by the two groups, 29,800 for the schizophrenic patients and 30,000 for the 


freshman subjects, compared with data from Horn (7) on children 


/ 











Children 


Schizophrenics 














Freshmen 
Mean Range Mean Range Mean Range 
Nouns 50.65 42.2-59.1 13.04 10.40-16.63 15.39 12.67-18.53 
Pronouns 2.25 @- 3.6 22.68 19.33-24.73 17.96 14.40-20.40 
Verbs 27.75 16.9-38.6 26.28 24.27-30.47 22.95 20.50-24.47 
Adverbs 65 2.5- 8.8 11.54 7.00-17.97 10.16 8.87-11.20 
Conjunctions 1.5 3- 2.7 6.53 4.-10- 8.77 8.83 7.33-11.40 
Prepositions I.I .6- 1.6 7.48 4.30-10.00 10.00 8.80-11.10 
Interjections .6 Oo — 1.2 2.64 .53- 4-43 1.26 .47- 2.00 
Adjectives 13.45 10.1-16. 9.85 8 .60-12.40 13.48 II.43-16.40 





showing the range of percentages on 
parts of speech that 11 investigators have 
found in children’s language, as com- 
pared to the percentages for the schizo- 
phrenics and freshmen found in this 
study. For case of comparison the experi- 
menter averaged these ranges, each study 
having been done on only one child. 
Here we immediately note some striking 
differences. The children used approxi- 
mately three to four times as many 
nouns as either the schizophrenics or 
freshmen. They used eight to ten times 
fewer pronouns, about half as many ad- 
verbs, four to six times fewer conjunc- 
tions, seven to ten times fewer preposi- 
tions, two to four times fewer exclama- 
tions, about the same number of adyjec- 
tives as did the freshmen (hence more 
than the schizophrenics), and about the 
same number of verbs as did the schizo- 
phrenics (hence fewer than did the fresh- 
men). Again no conclusive comparisons 
can be made because of the probably 
varying procedures used in making the 
grammatical analyses. 

If reference is made also to the French, 
Carter, and Koenig data, one might con- 
clude that while the relative proportions 
of the various parts of speech change 
greatly from childhood to the adult 
level, the differences among various sam- 
ples of adults are much smaller. Cer- 





tainly there is no apparent tendency for 
the schizophrenic patients to regress to- 
ward the childhood level with respect to 
the general grammatical construction of 
their language, unless it might be in re- 
gard to more frequent use of verbs. 

3. Word frequencies. Table 6 gives a 
list of the 100 most frequently used 
words for the schizophrenic patients and 
the freshmen, respectively, the list for 
the latter having those words which are 
common to both lists arranged in order 
of frequency, while the list for the schizo- 
phrenic patients has the words corre- 
sponding to those of the freshmen ar- 
ranged in order of sequence regardless 
of frequency. The 21 words in each of 
the two groups not common to both lists 
are arranged at the bottom of the table 
in order of frequency. Several interesting 
differences the list for the 
schizophrenic patients and that for the 
freshmen can be noted in regard to the 
frequencies for various words. For ex- 
ample, the schizophrenics used not al- 
most twice as many 
freshmen. In 


between 


times as did the 
addition, no and never 
occur in the list for the schizophrenics, 
while no other clearly negative words 
occur in the first 100 words for the fresh- 
men. Hence, we have the schizophrenics 
using these negative words 1,087 times 
to 484 times for the freshmen, the former 
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List of 100 words most frequently used by schizophrenics and freshmen. The first 79 words common 
to both lists are arranged in descending rank order according to frequency of usage by 
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TABLE 6 


freshmen. The remaining 21 words not common to both lists are arranged in order 
of frequency for the two groups at the end of the table 
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Freshmen Schizophrenics 
* Part of Part of 

Word Speech Freq. Word Speech Freq. 

1. the art. 1140 the art. 735 
2. and conj. 1113 and conj. 785 
4 3 pron. 924 I pron. 2501 
4. a art. 788 a art. 356 
5. to prep. 779 to prep. 035 
6. is verb 629 is verb 580 
7. it pron. 623 it pron. 729 
8. of prep. 612 of prep. 4106 
g. that pron, 599 that pron. 633 
Io. you pron. 562 you pron. 392 
Ir. not adv. 484 not adv. 942 
12. in prep. 396 in prep. 266 
13. he (He) pron. 347 he (He) pron. 244 
14. that conj. 327 that conj. 172 
15. have verb. 305 have verb 339 
16. do verb. 304 do verb 638 
17. they pron. 276 they pron. 321 
18. well interj. 271 well inter}. 565 
Ig. Was verb 270 was verb 412 
20. are verb 238 are verb 136 
21. if conj. 234 if conj. 164 
22. she pron. 220 she pron. 127 
23. we pron. 218 we pron. 79 
24. but conj. 211 but conj. 173 
25. or conj. 204 or con). 150 
26. just adv. 177 just adv. 190 
27. for prep. 175 for prep. 128 
28. there adv. 168 there adv. 163 
29. with prep. 165 with prep. 98 
30. would verb. 164 would verb 226 
31. had verb 159 had verb 212 
32. what (uh?) pron. 155 what (uh?) pron. 297 
33. very adv. 154 very adv. 46 
34. think verb 147 think verb 131 
35. oh interj. 143 oh inter). 125 
36. about prep. 141 about prep. 133 
37. know verb 139 know verb 496 
38. on prep. 138 on prep. 109 
39. get verb 125 get verb 120 
40. at prep. 117 at prep. 73 
41. out adv. 115 out adv. 95 
42. will verb 113 will verb 52 
43. people noun 11I people noun 74 
44. something noun 108 something noun 86 
45. them pron. 108 them pron. 66 
46. this pron. 100 this pron. 85 
47. one pron. 99 one pron. 72 
48. me pron. 96 me pron. 272 
49. up adv. 93 up adv. 71 
50. when conj. 93 when conj. 114 
51. might verb 89 might verb 47 
52. then adv. 85 then adv. 100 
S.. con)j. 84 as conj. 46 
54. things noun 84 things noun 80 
55. time noun 83 time noun 61 
56. because conj. 82 because conj. 149 
57. can ver 78 can verb 75 
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TABLE 6 (Continued) 


























































Freshmen Schizophrenics 
Part of Part of 
Word Speech Freq. Word Speech Freq. 
58. were verb 76 were verb 58 
59. say verb 75 say verb 104 
60. good adj. 75 good adj. 47 
61. him (Him) pron. 74 him (Him) pron. 57 
62. go verb 71 go verb 56 
63. my pron. 71 my pron. 286 
64. cannot verb 70 cannot verb 93 
65. did verb 7O did verb 158 
66. like prep. 69 like prep. 82 
67. all adj. 68 all adj. 53 
68. so adv. 62 sO adv. 75 
69. see verb 62 see verb 47 
70. am verb 61 am verb 167 
71. one adj. 59 one adj. 64 
72. some adj. 59 some adj. 54 
73. anything pron. 59 anything pron. IIo 
74. could verb 58 could verb 121 
75. got verb 56 got verb 72 
76. want verb 52 want verb 62 
77. been verb 52 been verb 67 
78. way noun 48 way noun 58 
79. Means verb 48 means verb 93 
80. his (His) pron. 121 yes (uh huh) adv. 173 
81. person noun 118 be verb 145 
82. an art. 103 said verb 109 
83. has verb 102 no (hunh uh) adv. 96 
84. who pron. 102 why interj. 89 
85. her pron. 76 suppose verb 85 
86. so conj. 74 now adv. 82 
87. by prep. 71 guess verb 73 
88. let noun 65 here adv. 73 
89. from prep. 65 any adj. 71 
go. other adj. 63 thought verb 70 
gi. example noun 63 mean verb 66 
92. going verb 62 sir noun 65 
93. quite adv. 61 thing noun 56 
94. your pron. 58 too adv. 56 
95. which pron. 57 all noun 53 
96. does verb 54 never adv. 49 
97. always adv. 54 understand verb 49 
98. us pron. 50 little adj. 47 
99. then conj. 49 right adj. 45 
100. course noun 48 tell verb 43 















group using them about two and one- 
half times more than the freshmen, when 
only the 100 most frequently used words 
are considered. Instead of the never used 
by the schizophrenics ,the freshmen used 
always about an equal number of times. 
Another interesting item is that the 
freshmen used very over three times as 
often as did the schizophrenics. When 
the verbs among these 100 most fre- 
quently used words for the group were 





considered, it was found that the schizo- 
phrenic patients used eight past tense 
verbs a total of 1158 times while the 
freshmen used six such verbs only 683 
times. It is interesting to note, also, that 
two verbs carrying the connotation of in- 
decision, suppose and guess, occur among 
the 100 most frequently used words of 
the schizophrenics for a total of 158 
times, while no such words occur in the 
comparable list for the freshmen. 
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A more detailed comparison of fre- 
quencies for various words used by the 
schizophrenic group and by the fresh- 
man group possibly would show several 
interesting and differential facts. A con- 
sideration of the qualitative aspects of 
some of the words used by the two 
groups would also provide interesting 
material. 

Because of the tendency shown in the 
TTR analysis for the schizophrenic pa- 
tients to repeat words more frequently 
than do the freshmen, Fig. 1 is presented 
to show what proportion the 100 most 
frequently used words constituted of the 
total number of words for the two 
groups. We may refer to this as propor- 
tional vocabulary. The frequencies for 
each consecutive five words, starting with 
the most frequently used word, were 
added cumulatively for each group, and 
these successive cumulative frequencies 
were expressed as fractions of the total 
number of words. The curves show that 
the patients consistently use a smaller 
number of different words to represent 
any given percentage of the total number 
of words. For example, the schizophrenic 
group use only 33 words to make up 50 
per cent of the total number of words, 
while the freshman group use 46 words 
to arrive at the same percentage. The 
entire 100 most frequent words consti- 
tute 68.32 per cent of the total number 
of words for the schizophrenics as a 
group, and 62.91 per cent for the fresh- 
men. Superimposed on these curves is a 
similar curve taken from the French, 
Carter, and Koenig (6) study, indicating 
that the 100 most frequently used words 
in the telephone conversations analyzed 
formed 75 per cent of the total number 
of words. The curve on written material 
was also given by French, Carter, and 
Koenig, and was taken by them from 
Dewey (5). According to it, the 100 most 


frequently used words in written mate- 
rial form only 56 per cent of the total 
number of words used. A consideration 
of all four curves shows that the tele- 
phone conversation and written English 
represent the extremes in this factor of 
repetition, or number of types constitut- 
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Fic, I. Curves showing the cumulative percent- 
ages of the total words for the 100 most fre- 
quently used words. A, telephone conversation 
(0); B, schizophrenic subjects; C, freshman sub- 
jects; D, written material (5). 


ing a given percentage of the total num- 
ber of tokens, the telephone conversa- 
tion being the most repetitious and the 
written English the least. This might be 
expected from the stereotyped, truncated 
nature of telephone conversation as com- 
pared to the reflective style of written 
English in which a premium is placed 
on variety. That the curves for the two 
groups considered in this study should 
fall in between these extremes, that for 
the schizophrenic group more nearly ap- 
proximating that for the telephone con- 
versation and the curve for the freshman 
group being closer to the one for the 
written English, might also be expected, 
considering the repetitious nature of the 
schizophrenic speech. The freshmen ap- 
pear to have been more successful in in- 
troducing variety and flexibility into 
their spoken language. The lower end 
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of the curves indicates the interesting 
fact that the curve for written English 
overlaps the other three and is higher 
for the first five words or so. This might 
be explained as being due to the com- 
pleteness of written English as compared 
to conversations, the articles and the con- 


three times as often as do the schizo- 
phrenics; the second person pronoun, 
plural and singular (you, your, yours, 
yourself, thee, thou), almost twice as fre- 
quently as do the schizophrenics; and 
the third person pronoun, singular and 
plural (he, his, him, himself, she, her 


TABLE 7 
Relative frequency of use of the different personal pronouns expressed as percentage of the total 
number of words for the two groups, 29,800 for the schizophrenic patients 
and 30,000 for the freshmen 








Schizophrenics 





Freshmen 





N 





% N 





Ist person sing. 3104 
Ist person plural 102 
2nd person sing. and plural 429 
3rd person sing. and plural 1645 


10.42 1107 
32 315 
1.44 643 
~52 1923 





junctions and prepositions being used by 
writers probably more than by speakers. 
An examination of Dewey's list shows 
that the five most frequently used words 
in the written English samples were the, 
of, and, to, and a. 

Another analysis that suggests itself, 
because of the schizophrenic’s self-preoc- 
cupation and his tendency to ignore his 
environment, is the relative frequency 
of referrals to self and of referrals to 
others found in the language of the two 
groups. This analysis was made by com- 
puting the percentage of the total num- 
ber of words represented by the different 
personal pronouns. Table 7 shows the 
results of this computation. The most 
striking fact in this table is that refer- 
ences to self, using some form of the first 
person singular pronoun (J, my, mine, 
me, myself), make up 10.42 per cent of 
the total number of words for the schizo- 
phrenic group, while they represent only 
3.69 of the total for the freshmen. On 
the other hand the freshmen use the 
first person plural pronoun in its vari- 
ous forms (we, our, ours, us, ourselves) 


hers, herself, it, its, itself, they, thetr, 
theirs, them), almost 20 per cent more 
often. 

The schizophrenic patients used a 
total of 14 neologisms, or coined words. 
These words are banoon, d-s, d-t, dokey, 
g-m, g-o-d-t, okey-dokey, oke, pard, 
p-a-r-d, strob,  striked, 
woozy, adjects. Neologisms were not 
found in the freshmen samples. 


recognization, 


IV. SUMMARY AND CONCLUSIONS 

Three-thousand-word language sam- 
ples were obtained from each of ten 
schizophrenic patients, five males and 
five females, and ten University of Iowa 
freshmen, four males and six females, the 
latter ranking above the goth percentile 
on the Composite Score of the Iowa 
Qualifying and Placement Examina- 
tions. An interview situation was em- 
ployed, involving the interpretation of 
14 proverbs, the interviews being re- 
corded by an electric dictaphone tech- 
nique without the subjects’ knowledge. 

A word count was then made for each 
language sample, each word being tabu- 
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lated according to its frequency in con- 
secutive 100-word segments and accord- 
ing to its grammatical usage. Three 
types of analysis were made: (1) the type- 
token ratio, computed by dividing the 
number of different words in each 100- 
word segment by the total 100 words; 
(2) grammatical analysis; and (3) word 
frequencies. 

1. When the t-test for related meas- 
ures was applied to the language samples 
for both groups by dividing at random 
the 30 I'TR’s for each subject into two 
sets and finding the group mean for each 
half, it was found there was no signifi- 
cant difference between the two means 
either for the schizophrenics or for the 
freshmen. 

2. When the ratio of the variance of 
the distribution of the standard devia- 
tions of the segmental TTR’s for the 
schizophrenic patients to that of the 
freshmen was computed, the resulting F 
value was not significant, indicating that 
the schizophrenic patients did not vary 
more from segment to segment than did 
the freshmen. 

3. The mean TTR’s of the schizo- 
phrenic patients were generally lower 
than were those of the freshmen, and the 
range for the patients was much greater. 

4. The group mean TTR of the schiz- 
ophrenic patients was found to be sig- 
nificantly lower than the group mean 
TTR for the freshmen. 

5. It is probable that a positive cor- 
relation exists between the TTR and the 
intellectual level, according to previously 
reported findings, and judging by the 
indicated relationship between the 
T'TR’s of the patients and their prob- 
able intellectual levels when both were 


ranked in descending order. However, 
there were differences between the mean 
TTR’s for the schizophrenic patients 
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who ranked highest intellectually and 
most of the freshmen. 

6. When the t-test was applied to test 
the difference between the two groups 
in terms of the relative frequency of 
usage of the eight grammatical cate- 
gories, expressed as percentages of the 
total number of words used, it was found 
that the schizophrenic patients used sig- 
nificantly fewer nouns, conjunctions, 
prepositions, adjectives, and articles than 
did the freshmen, and significantly more 
pronouns, verbs, and interjections. 

7. The F ratio, involving the variances 
of the distributions of percentages (based 
on total words per sample) for each 
grammatical category for the two groups 
revealed that the schizophrenic patients 
Showed significantly greater variability 
than did the freshmen in the use of ad- 
verbs, prepositions, and interjections. 

8. Comparison of the relative propor- 
tions of the various parts of speech found 
in this study with those given in another 
study on telephone conversation, for pre- 
sumably normal adults, indicates a very 
close approximation between the. per- 
centages of nouns, pronouns, and verbs 
used in telephone conversation and 
those. used by the freshman group. ‘The 
procedure used in the former study for 
classifying these three parts of speech 
was quite similar to that used in the 
present study. The procedures for clas- 
sifying the prepositions and conjunc- 
tions, and the articles also apparently 
were similar, but the percentage for the 
former was considerably lower for the 
telephone conversation than for the 
freshman language, and the percentage 
of articles was slightly lower. The per- 
centages of adjectives and adverbs were 
also considerably lower for the telephone 
conversation than for either the schizo- 
phrenic or freshman samples, and the 


























percentage of interjections was a great 
deal higher, but the procedures for the 
classification of these two groups of 
words differed considerably in the two 
studies. 

The most definite differences between 
the schizophrenic patients and the nor- 
mal adults in this and the other study 
lie in the fact that the patients used pro- 
portionately more pronouns and verbs, 
and proportionately fewer nouns and ar- 
ticles. 

g. A general comparison with similar 
data on children under six and one-half 
years of age showed several marked dif- 
ferences between the percentages on the 
parts of speech for the children and 
those for the two groups in this study. 
The children used many more nouns 
and many fewer pronouns, adverbs, con- 
junctions, prepositions and interjections 
than either the schizophrenic or fresh- 
man group. In the percentage of verbs 
the children more closely resembled the 
schizophrenic group, and their percent- 
ages of adjectives was nearly the same as 
for the freshman group. 

10. Assuming that the probably differ- 
ent procedures in the grammatical anal- 
yses of the three studies permit general 
comparisons, it would appear that while 
the relative proportions of the various 
parts of speech change greatly from 
childhood to the adult level, the differ- 
ences among various samples of adults 
are much smaller. There was little evi- 
dence from this analysis that schizo- 
phrenia constitutes a regressive tend- 
ency, except for the more frequent use 
of verbs, the other findings for the chil- 
dren and schizophrenics, respectively, 
being decidedly different. 

11. When a list of the 100 words most 
frequently used by the schizophrenics 
and by the freshmen was made, it was 
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found each list had 21 words not com- 
mon to the other. 

12. The total frequencies for these 100 
most frequently used words constituted 
68.32 per cent of the total number of 
words used by the schizophrenics and 
62.91 per cent of those used by the fresh- 
men, the schizophrenics consistently 
using a smaller number of different 
words to make up any given percentage 
of the total up to this figure. For the 
schizophrenics 33 different words (types) 
constituted 50 per cent of their total sam- 
ple of 29,800 words (tokens); for the 
normals 46 types constituted 50 per cent 
of their 30,000 tokens. 

13. A comparison of the relative pro- 
portion of referrals to self and referrals 
to others, as indicated by the use of per- 
sonal pronouns by the two groups, shows 
that the schizophrenics used more first 
person singular pronouns, and fewer first 
person plural, second person plural and 
singular, and third person plural and 
singular pronouns than did the fresh- 
men, J, my, mine, me, and myself repre- 
sented 10.42 per cent of the total num- 
ber of words for the schizophrenics, and 
only 3.69 per cent of the total words for 
the freshmen. 

14. Several interesting differences in 
the frequencies of occurrence of specific 
words among these 100 most frequently 
used words were noted, such as the fact 
that negative words (not, no, and never) 
have a frequency two and one-half times 
larger in the schizophrenic list than in 
the freshman list, and that verbs in the 
past tense had a frequency a little less 
than twice as large in the schizophrenic 
list as in the freshman list. 

The conclusion can be stated that the 
measures used do make possible the 
quantitative expression of certain differ- 
ences among samples of spoken lan- 
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guage. Statistically significant differences 
between schizophrenic language and the 
language of superior university fresh- 
men, as these types of language were 
here sampled, were indicated by the 
measures of vocabulary extent and ‘flexi- 
bility’, and of grammatical structure. 
The measures of word frequency were 
also suggestive of some possibly impor- 
tant differences between the two groups. 

These findings are to be evaluated 
with clear awareness that they may not 
be due entirely to the schizophrenia, 
since, as was explained in the Introduc- 
tion, there were necessarily differences 
between the two groups with regard, par- 
ticularly, to intelligence and scholastic 
training, and the relevance of these dif- 
ferences cannot, at this stage of investiga- 
tion, be clearly judged. The degree to 
which such ‘intellectual’ factors are re- 
lated to the language measures employed 
is not yet known; and the problem of 
measuring the intelligence of psycho- 
pathological individuals is by no means 
simple. Insofar as any conclusions may 
be drawn about ‘schizophrenic language’ 
on the basis of this study, they would 
appear to suggest the possibility that 
such language differs from the language 
of ‘normal’ persons in being (a) less 
highly differentiated in structure—the 
ratio of different words (types) to total 
words (tokens) is lower, as shown by the 


analyses in terms of type-token ratio and 
proportional vocabulary; (b) more nega- 
tively toned; (c) indicative of preoccupa- 
tion with the past, as shown by relatively 
more past tense verbs; (d) indicative of 
more self-reference, as shown by more 
frequent occurrence of self-reference 
terms in the first person singular pro- 
noun class; (e) characterized by a slight 
tendency toward the use of neologisms; 
(f) featured by a probable peculiarity of 
grammatical structure, represented by 
relatively more pronouns and verbs and 
fewer nouns and articles, which might 
possibly be suggestive of excessive self- 
preoccupation and ‘instability’. More- 
over, such comparison as could be made 
of the ‘schizophrenic language’ and that 
of children (7) provided little ground for 
the view that schizophrenia constitutes 
a regression to childhood behavior pat- 
terns, in that the language of the schizo- 
phrenics, as measured, bore no striking 
resemblance to that of the children, ex- 
cept possibly in the proportionate num- 
ber of verbs. 

Again, it is to be emphasized that this 
study was designed primarily to explore 
the possibilities of language measure- 
ment. From this point of view, its results 
may be regarded as definitely promising. 
Any conclusions concerning the nature 
of ‘schizophrenic language’ are advanced 
only for their suggestive value. 
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II1. THE QUANTITATIVE DIFFERENTIATION OF SAMPLES OF WRITTEN LANGUAGE! 


MARY BACHMAN MANN 
University of lowa 


I. INTRODUCTION 


HIS STUDY is part of a previously de- 
‘Leia program of research con- 
cerned with the general problem of 
language behavior (16). 

The present investigation is concerned 
primarily with the objective of develop- 
ing reliable and differentiating measures 
of language behavior, and, to a limited 
extent, with determining the intercor- 
relation of the measures, their relation 
to other pertinent variables, and with in- 
dicating the normal characteristics of 
language behavior as contrasted with 
disorder in such behavior. 

The scientific study of language be- 
havior has been carried on by many 
investigators, among them Piaget (18), 
Cameron (5), Thorndike (22), Horn 
(13), Zipf (25), Carroll (10), Skinner (20), 
Jersild and Ritzman (14), Balken and 
Masserman (2), and Fairbanks (12), to 
mention only a few.? 

None of the previous investigators has 
been precisely concerned with the par- 
ticular issues around which the present 
study is centered. In the first place, the 


‘This study was done in the Department of 
Psychology at the State University of Iowa as a 
dissertation in partial fulfillment of the require- 
ments for the degree of Doctor of Philosophy. 
The study was directed by Wendell Johnson, 
and is part of a program of research on lan- 
guage behavior. The writer is grateful to Dr. 
Andrew H. Woods, Director, and the staff of 
the Iowa State Psychopathic Hospital, and Dr. 
Leonard P. Ristine, Superintendent, and the staff 
of the Mt. Pleasant State Hospital, for their co- 
operation in securing subjects for the investiga- 
tion. 

* The reader who is interested in the study 
of language from the standpoint of vocabulary 
and word lists will find an excellent summary in 
Fries, Charles C. and Traver, A. Aileen, English 
Word Lists, American Council on Education, 
Washington, D.C., 1940, pp. 109. 
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present study is strictly quantitative, and 
this fact serves to differentiate it from a 
considerable proportion of previous in- 
vestigations of language. Secondly, this 
study is concerned with the language be- 
havior of specified individuals, a fact 
which differentiates it from practically 
all of the word-frequency studies such as 
those of Thorndike and Horn, in which 
large samples of language drawn from a 
variety of sources were studied but with 
no attention given to the characteristics 
of the language of individuals. Thirdly, 
some of the measures used in the present 
study, particularly the type-token ratios, 
have not been employed, as they are here 
used, in any previous studies with the 
exception of the one by Fairbanks (12) 
which may be regarded as a companion 
study to this one. 

What was desired, for purposes of this 
particular study, was a sampling of the 
language of persons who could be re. 
garded definitely as psychopathological, 
but who could nevertheless produce writ- 
ten language, and a sampling of the lan- 
guage of persons who could be regarded 
as definitely superior in verbal ability, 
but who might not be regarded as ‘ver- 
bal specialists’, such as outstanding nov- 
elists, scientific writers, etc. The study is 
concerned, first of all, with the specific 
problem of whether and in what respects 
‘adequate’ and ‘inadequate’ language 
might be differentiated quantitatively. 
The problem of ascertaining the par- 
ticular factors responsible for any dem- 
onstrated differences between the ade- 
quate and inadequate language is sec- 
ondary to the main investigation, but it 
has been considered in some degree. 
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Adults schizophrenic patients were se- 
lected as the subjects from whom samples 
of ‘inadequate’ language were to be ob- 
tained. Other specific types of subjects 
might have been chosen; subjects might 
have been selected, for example, solely 
on the basis of educational level, or of 
intelligence test scores. Aphasics might 
have been used in order to obtain ‘in- 
adequate’ language. Aphasics, however, 
might be expected to produce language 
‘inadequate’ in some relatively rare 
sense. And schizophrenics were preferred 
to persons mainly characterized by low- 
grade test-intelligence, or by low scho- 
lastic achievement, because insofar as 
their language is ‘inadequate’ it would 
appear to be so in a peculiarly significant 
sense from the standpoint of social ad- 
justment. Thus, in the case of schizo- 
phrenics, neuro-linguistic inadequacy, 
insofar as it may exist, may reasonably 
be judged to have a significance beyond 
that of the neuro-linguistic inadequacy 
involved in ‘simple’ low-grade ‘intelli- 
gence’. 

Having selected the subjects from 
whom the ‘inadequate’ language samples 
were to be obtained, the problem of se- 
lecting a contrasting group of subjects 
presented itself. This problem was essen- 
tially that of selecting subjects from 
whom relatively high-grade language be- 
havior might be expected, but who could 
be counted upon not to produce lan- 
guage that was highly ‘adequate’ in some 
relatively exceptional respect. Superior 
‘literary’ language, for example, was to 
be avoided. After due consideration, the 
decision was made to select subjects who 
were not noted as being talented in some 
exceptional linguistic respect, who were 
behaviorally and socially normal in the 
sense, at least, that they could function 
as freshmen in a large university, and 
who were neuro-linguistically superior 


in the sense that they scored relatively 
very high on a battery of largely verbal 
tests administered to them on the occa- 
sion of their entering the university 
which they were attending. 

The question might be raised as to the 
advisability of selecting ‘normal’ subjects 
matched with the psychotic patients with 
respect to such factors as ‘intelligence’, 
educational status, etc. The most impor- 
tant consideration in this connection is 
simply that such a procedure would 
probably have militated against the 
main purpose of the study, in that it 
would have made less likely the obtain- 
ing of two definitely differing samples 
of language. It was a primary considera- 
tion that two such samples be obtained 
if the problem of the quantitative dif- 
ferentiation of language samples was to 
be fruitfully investigated. A determina- 
tion of the respects in which language 
samples of the type were utilized might 
be quantitatively differentiated would 
appear to be basic to any study of the 
relation of specific factors, such as ‘intel- 
ligence’, for example, to measurable as- 
pects of language behavior.* 

The language obtained from the psy- 
chotic subjects used in this investigation 
definitely constitutes a sample of the 

*'The question as to whether the language of 
schizophrenics differs, insofar as it does, from 
the language of superior university freshmen, be- 
cause the schizophrenics are less “intelligent,” 
raises an extremely complicated issue. It is not to 
be ligthtly dismissed, for example, that the 
phrase “highly intelligent schizophrenic” may be 
in a basic sense self-contradictory. The fact that 
an “intelligence test’’ shows a schizophrenic to 
be superior mentally probably tells, from one 
point of view, as much about the test as it does 
about the patient. The schizophrenic offers a 
means of validating the test quite as definitely 
as the test offers a means of evaluating the pa- 
tient. A particularly pertinent answer to the test 
on which a schizophrenic scores a high “intelli- 
gence quotient” is that, when all is said, the 
schizophrenic is in custody. The issue is not a 
simple one by any means; further discussion of 


it, however, is hardly relevant to the present pur- 
poses. 
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language of schizophrenics, but whether 
its differentiating characteristics are due 
to ‘schizophrenia’ is a question, not with- 
out interest, but not of primary concern 
in this study, Of course, insofar as the 
differentiating characteristics of the 
schizophrenics’ language cannot be at- 
tributed to something else, it would ap- 
pear reasonable to regard them as due 
to, or as involved in, whatever may be 
designated by the term ‘schizophrenia’. 
The relation of test-intelligence and edu- 
cational level, at least, as well as that of 
sex, to the quantitatively expressible as- 
pects of the schizophrenics’ language has 
been ascertained to some extent in the 
present investigation. It is to be clearly 
understood that one is to be cautious, 
though not to the point of impotence, 
in drawing from this study any general- 
izations concerning ‘the language of 
schizophrenia’, since the study is de- 
signed primarily to yield generalizations 
with respect to another problem, namely 
that concerning the quantitative differ- 
entiation of samples of written language. 


II. STATEMENT OF THE PROBLEM 


This study is concerned with the fol- 
lowing specific problem: the quantita- 
tive differentiation of samples of pre- 
sumably adequate and inadequate writ- 
ten language, as obtained from superior 
university freshmen and schizophrenic 
patients, respectively, in terms of the fol- 
lowing specific measures: 


(1) The ratio of types (different words) to 
tokens (total words used). 

(2) The relative frequency of usage of cer- 
tain grammatical categories. 

(3) The ratios of the frequency of occur- 
rence of adjectives to verbs, adjectives 
to nouns, and adverbs to verbs, respec- 
tively. 

(4) The relative frequency of specific 
types, expressed as percentage of to- 
kens. 


Ill. PROCEDURE 


Two groups of adults served as sub- 
jects in this investigation: (1) twenty- 
four psychotic patients diagnosed as 
schizophrenic were selected to represent 
a group presenting psychopathological 
or inadequate language; (2) twenty-four 
superior university freshmen were se- 
lected to represent a group presenting 
relatively adequate language. A summary 
of the main characteristics of these two 
groups follows. 


At the time the data were secured the 
patients were all confined in the Mt. 
Pleasant State Hospital at Mt. Pleasant, 
Iowa. Thirteen of the twenty-four had 
been previously examined at Iowa State 
Psychopathic Hospital, Iowa City, and 
the diagnosis of schizophrenia made by 
the psychiatrists at the Iowa State Psy- 
chopathic Hospital had been confirmed 
by the staff at the Mt. Pleasant State Hos- 
pital. These particular schizophrenic pa- 
tients were selected because of the rela- 
tively maximum certainty of the diag- 
nosis, and the possibility of securing 
their cooperation in the proposed writ- 
ing situation, The patients, twelve male 
and twelve female, ranged in age from 
sixteen to forty-nine years, with an aver- 
age age of thirty-two years; four (one 
male and three females) have been mar- 
ried. The average duration of present 
confinement in the Mt. Pleasant State 
Hospital prior to their service as sub- 
jects for this investigation was three 
years and three months, the range being 
from one year to eight years. The aver- 
age duration of the illness, taken from 
the time of the first psychotic symptoms, 
as shown in the patient’s hospital rec- 
ord,* was five and one-half years, ranging 








*This must be considered as only an indica- 
tion of duration of illness since it is difficult if 
not impossible in many cases to determine when 
the illness began. In the case of the disease 
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from one year to eleven years. Prior to 
their commitment in the hospital the 
patients had been engaged in the follow- 
ing occupations: laborer, accountant, 
farm laborer, high school student, col- 
lege student, university law student, but- 
ton cutter, pharmacy clerk, school 
teacher, telephone operator, hospital 
maid, and housewife. The level of edu- 
cational attainment ranged from grade 
eight to college graduate; sixteen of the 
twenty-four were high school graduates 
and ten of those sixteen had some col- 
lege training. Of the fifteen patients for 
whom intelligence ratings were avail- 
able, the range in I.Q. points was from 
78 to 138, the mean I.Q. being gg. It 
should be pointed out that mere I.Q. 
scores on these patients have little mean- 
ing, and care must be exercised in inter- 
preting such scores. Where it was pos- 
sible to do so, a vocabulary® score, or a 
verbal scale I.Q., and a_ performance 
scale 1.Q. have been given. The intel- 
ligence tests were all administered by 
the hospital psychometrist and judg- 
ments as to probable classification are 
those of the psychometrist. Of the tests 
used, two were Wechsler-Bellevue Adult 
Scale; ten were Revised Stanford-Binet, 
Form L; one was Revised Stanford- 
Binet, Form M; and two were the 1916 
Stanford Revision of the Binet-Simon 
Test. 

Within the diagnosis of schizophrenia, 
twelve patients had been further clas- 
sified as hebephrenic, three as simplex, 





schizophrenia the onset is insidious, often ex- 
tending over a period of several years before 
definite psychotic symptoms appear and are diag- 
nosed. Furthermore, it is frequently difficult to 
ascertain from hospital records who first con- 
sidered the behavior abnormal and diagnosed it 
as psychotic. 

* According to Babcock (1) vocabulary is the 
best measure of the original intellectual level of 
the psychotic individual. 
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seven as paranoid, and two as catatonic. 
The following abstracts present data 
concerning the individual patients, ‘The 
information contained in these abstracts 
was taken from the hospital records for 
each patient. 

Case 1. Diagnosis: Schizophrenia, he- 
bephrenic type. A married female, thirty- 
eight years of age; completed college 
education and taught school one year 
after graduation before being married. 
First psychotic symptoms in 1931, pres- 
ent commitment to Mt. Pleasant State 
Hospital began in 1932, having previ- 
ously been institutionalized in private 
sanitariums on two occasions. Scored In- 
telligence Quotient of 107 on Wechsler- 
Bellevue Adult Scale, Verbal Scale 1.Q. 
112, and Performance Scale I.Q. 100. 
Classification by psychometrist: Above 
Average; some inefficiency. 

Case 2. Diagnosis: Schizophrenia, he- 
bephrenic type. A single male, thirty 
years of age; educated through tenth 
grade in high school; occupation before 
committed to hospital, none. First psy- 
chotic symptoms one year before com- 
mitment to Mt. Pleasant State Hospital 
in 1936. No previous commitments. No 
intelligence test results available. 

Case 3. Diagnosis: Schizophrenia, cata- 
tonic type. A single male, thirty years of 
age; educated through high school; pre- 
vious occupations, working in restaurant 
and drug store. First psychotic symptoms 
in 1929, present commitment to Mt. 
Pleasant State Hospital began in 1936, 
having been committed previously for 
short periods in 1929 and again in 1931. 
No intelligence test results available. 

Case 4. Diagnosis: Schizophrenia, para- 
noid type. A single female, aged twenty- 
seven years, educated through two years 
of college. First mental symptoms in 
1930, present episode began in 1938, hos- 
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pitalized at Iowa State Psychopathic in 
1939, then committed to Mt, Pleasant 
State Hospital and confined there since. 
Scored Intelligence Quotient of 138 on 
Revised Stanford-Binet, Form L, passing 
vocabulary at Superior Adult III level. 
Psychometrist commented that intellec- 
tual level was “‘very superior’’. 

Case 5. Diagnosis: Schizophrenia, he- 
bephrenic type. A single female, forty- 
eight years of age, graduated from high 
school and attended a teachers college 
one summer, ‘Taught school three years 
before first attack which was in 1916 
lasting for approximately one year, then 
did housework at home. Second and 
present attack began in 1938 when she 
was committed to Mt. Pleasant State 
Hospital, Scored Intelligence Quotient 
of g1 on Wechsler-Bellevue Adult Intel- 
ligence Scale, Verbal Scale 1.Q. 106, Per- 
formance Scale I.Q. 78..Psychometrist’s 
statement: ‘The patient’s intellectual 
development is average.” 

Case 6. Diagnosis: Schizophrenia, para- 
noid type. A single male, thirty-two years 
of age, graduated from high school. 
Worked as a laborer prior to commit- 
ment to Mt. Pleasant State Hospital in 
1938. The onset of the present episode 
was gradual, believed to have begun six 
or seven years before time of commit- 
ment. Scored Intelligence Quotient of 
111 on Revised Stanford-Binet, Form L; 
vocabulary score, high average. Psy- 
chometrist’s statement: “There is noth- 
ing remarkable about his performance. 
It was consistently good and warrants a 
classification of High Average Adult.” 

Case 7. Diagnosis: Schizophrenia, sim- 
ple type. Single female, twenty-nine years 
of age, graduated from high school and 
worked successfully as a telephone opera- 
tor for six years. First psychotic symp- 
toms manifested in 1934, committed to 


Mt. Pleasant State Hospital in 1937 and 
confined there continuously since. Scored 
Intelligence Quotient of 78 on 1916 Re- 
vision of Stanford-Binet Test. Believed 
by psychometrist to reveal marked de- 
terioration from an average intellectual 
development. 

Case 8. Diagnosis: Schizophrenia, para- 
noid type. A single male, thirty-two years 
of age, educated through tenth grade at 
fifteen years, worked as a laborer prior 
to commitment in Mt. Pleasant State 
Hospital in 1939. First psychotic symp- 
toms twelve to eighteen months before 
commitment and had spent six weeks in 
a private sanitarium. Scored Intelligence 
Quotient of 101 on the Revised Stan- 
ford-Binet, Form L, passed vocabulary 
test at Average Adult level. Classifica. 
tion by psychometrist: Average. 

Case 9. Diagnosis: Schizophrenia, he- 
bephrenic. A single female, eighteen 
years of age; present mental episode oc- 
curred during senior year of high school; 
confined to Mt. Pleasant State Hospital 
in 1939. Scored Intelligence Quotient of 
108 on Revised Stanford-Binet, Form 
M. In this she showed a superior vocab- 
ulary. Classification: Average. 

Case 10, Diagnosis: Schizophrenia, 
catatonic type. A single male, thirty-two 
years of age. Graduated from high school 
and attended college two and one-half 
years; confined to Mt. Pleasant State 
Hospital in 1932. Scored Intelligence 
Quotient of 101 on Revised Stanford- 
Binet, Form L. Classification: Average. 

Case 41. Diagnosis: Schizophrenia, 
paranoid type. A married male, aged 
thirty-three years; educated through 
high school and two and one-half years 
of college; occupation, accountant. First 
symptoms in 1934, confined to Mt. Pleas- 
ant State Hospital since early in 1939. 
Scored Intelligence Quotient of 108 on 
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1916 Stanford Revision of the Binet- 
Simon Test. Classification: Average. 

Case 12. Diagnosis: Schizophrenia, 
hebephrenic type. A single female, twen- 
ty-three years of age; educated through 
high school, and one semester of college. 
First psychotic symptoms in 1935, con- 
fined to Mt. Pleasant State Hospital con- 
tinuously since June, 1938. Record of a 
Stanford-Binet Test given in 1927 with 
a C.A. of 11 indicated an I1.Q. of 136; 
Stanford Revision of the Binet-Simon 
Test, Form L, administered in 1938 
yielded an I1.Q. of 97, vocabulary, 84. 
Psychometrist commented: “Vocabulary 
indicates a previous very superior level.” 

Case 13. Diagnosis: Schizophrenia, 
hebephrenic type. A single male, sixteen 
years of age. Psychotic symptoms began 
while he was in the ninth grade in high 
school. Committed to Mt. Pleasant State 
Hospital in 1939. Test results: Wechsler- 
Bellevue Adult Intelligence Scale, I.Q. 
77, Verbal Scale I.Q. 65, Performance 
Scale 1.Q. 95; Revised Stanford-Binet, 
Form L, 1.Q. 84, vocabulary Average 
Adult. Classification: Formerly average— 
low average—poor school achievement. 

Case 14. Diagnosis: Schizophrenia, 
hebephrenic type. A single female, aged 
thirty-six years; educated through high 
school and two years of college. First psy- 
chotic symptoms in 1932, hospitalized at 
Iowa State Psychopathic Hospital in 
1933, then committed to Mt. Pleasant 
State Hospital and there since. Scored 
Intelligence Quotient of 83 on Revised 
Stanford-Binet, Form L, passing vocab- 
ulary test at Superior Adult II level. 
Judged by psychometrist to have origi- 
nally been “at least high average”. 

Case 15. Diagnosis: Schizophrenia, 
simple type. A single male, twenty-five 
years of age, educated through eighth 
grade. Worked as a farm laborer. First 
psychotic symptoms appeared one month 


before commitment to Mt. Pleasant State 
Hospital in 1939. No intelligence test re- 
sults available. 

Case 16. Diagnosis: Schizophrenia, 
simple type. A single male, twenty-seven 
years of age; educated through high 
school and two years of junior college, 
then entered University Law School, 
where he was a good average student. 
First symptoms in 1934, at which time 
he was examined at Iowa State Psycho- 
pathic Hospital. Confined at Mt. Pleas- 
ant State Hospital since 1938. Results of 
the Revised Stanford-Binet Test, Form 
L, given in 1938, show an Intelligence 
Quotient of 87, vocabulary test high. 
Classification: Average. Shows deteriora- 
tion from a probably superior intellec- 
tual development. 

Case 17. Diagnosis: Schizophrenia, 
paranoid type. A single male, forty-nine 
years of age; educated through eighth 
grade. Occupation had been button cut- 
ter. First psychotic symptoms in 1930 
when he was committed in Mt. Pleasant 
State Hospital for a short time, released, 
and then re-committed in 1936. No intel- 
ligence test results available. 

Case 18. Diagnosis: Schizophrenia, 
hebephrenic type. A married female, 
thirty-eight years of age, educated 
through high school and two summer 
sessions at college. ‘Taught school before 
and after marriage. First psychotic symp- 
toms four months before commitment to 
Mt. Pleasant State Hospital in 1935. No 
intelligence test results available. 

Case 19. Diagnosis: Schizophrenia, 
hebephrenic type. A single male, twenty- 
one years of age, educated through high 
school. Worked as a laborer before com- 
mitted to hospital. Examined at Iowa 
State Psychopathic Hospital and hos- 
pitalized for seven months in 1936. Com- 
mitted to Mt. Pleasant State Hospital in 
1939. Scored Intelligence Quotient of 
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101 on Revised Stanford-Binet Test, 
Form L. Classification: Average. 

Case 20. Diagnosis: Schizophrenia, 
hebephrenic type. A single female, forty- 
one years of age, educated through high 
school and two years at junior college. 
First psychotic symptoms in 1930, con- 
fined to Mt. Pleasant State Hospital for 
eight months. Re-entered the same hos- 
pital in 1934 and confined there con- 
tinuously since that time. No intelli- 
gence test results available. 

Case 21. Diagnosis: schizophrenia, 
paranoid type. A single male, thirty-five 
years of age, educated through high 
school and occupied as a pharmacy 
clerk. First psychotic symptoms in 1938, 
committed to Mt. Pleasant State Hos- 
pital in 1939. No intelligence test results 
available. 

Case 22. Diagnosis: Schizophrenia, 
paranoid type. A married female, aged 
thirty-two years, educated through elev- 
enth grade at seventeen, worked as a 
telephone operator until her marriage. 
First psychotic symptoms in 1938; spent 
two months in a private sanitarium early 
in 1939, and committed to Mt. Pleasant 
State Hospital in May, 1939. Scored In- 
telligence Quotient of 86 on Revised 
Stanford-Binet, Form L, vocabulary Av- 
erage Adult. Classification: Dull Nor- 
mal. 

Case 23. Diagnosis: Schizophrenia, 
hebephrenic type. A single female, aged 
thirty-five years, educated through high 
school and spent several years in a con- 
vent; had also been occupied as a maid 
in a hospital. Admitted to Mt. Pleasant 
State Hospital for the first time in 1922 
and discharged in 1932, re-admitted in 
1936 and has remained there continu- 
ously since that time. No intelligence test 
results available. 

Case 24. Diagnosis: Schizophrenia, 
hebephrenic type. A single female, forty- 


two years of age, educated through 
eighth grade. First psychotic symptoms 
in 1931, committed to Mt. Pleasant State 
Hospital in 1934. No intelligence test re- 
sults available. 

The individuals comprising the sec- 
ond group were freshmen students at the 
State University of Iowa selected on the 
basis of their scores on the Iowa Qualify- 
ing and Placement Examinations given 
in September, 1939.6 They all ranked 
from the goth to the ggth percentile on 
the Composite Score of the examina- 
tions, the percentiles being based on the 
scores made by the freshmen students 
taking the examinations that year. An 
unpublished study by Mitchell (17) in- 
dicated a correlation of .76 between the 
Intelligence Quotients of sixty-six fresh- 
men, as scored on the Revised Stanford- 
Binet, Form L, and the Composite Scores 
on the Iowa Qualifying and Placement 
Examination, the average Intelligence 
Quotient being 122. The freshmen used 
in the present study may be regarded as 
generally comparable, although some- 
what superior in terms of the test scores 
in question, to Mitchell’s freshmen stu- 
dents. 

Of the twenty-four freshmen, twelve 
were male and twelve were female; they 
ranged in age from seventeen years, five 
months to twenty-three years, one month. 
They came from homes in which the fol- 


lowing occupations were represented by — 


the wage-earners in the families: farmer, 
railroad engineer, jeweler, life insurance 
agent, plumber, piano tuner, attorney, 
professor of physiology, switchman, 
banker, shoe clerk, assistant postmaster, 
school teacher, real estate salesman, 


* Three of the twenty-four freshman subjects 
were taken from the entering class of September, 
1938, and one from the entering class of Septem- 
ber, 1940. In each case the subject wrote while 
he was a freshman. 
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cashier of bank, electrician, and clerical 
worker, 

Written language samples of 2800 
words in length were obtained from all 
of the subjects in the following manner. 
‘These instructions were read to the sub- 
ject: “You are to write a story of your 
life. Start at the beginning and write it 
just as you remember things. Any words 
will do. Even things that may seem un- 
important to you should be written and 
especially things that have made a dif- 
ference in your life. No one else will see 
what you have written.” Then a copy of 
the instructions was given to the subject 
so that he could refer to them again. 
Each subject was told that his story 
should be at least 2800 words in length. 
When a subject did not write enough or 
asked further questions, instructions 
were continued in the above terms, or 
neutral comments were made. With most 
of the subjects more than one sitting was 
necessary in order for them to write 
samples of the length required. 

In order to secure the written lan- 
guage samples, the patients were taken 
into a room off the ward in the hospital, 
and the freshmen were asked to come to 
a conference room in one of the univer- 
sity buildings. The writer secured the 
data from all subjects except the male 
patients, from whom the language sam- 
ples were obtained by a male attendant 
in the hospital. Consistently undisturb- 
ing conditions were maintained insofar 
as was possible, and to a practically suf- 
ficient degree, while the samples were 
being written. Not more than six sub- 
jects were writing at the same time in a 
large-sized room, and the average total 
time required of the patients to write 
the sample of the required length was 
approximately eight hours, while the 
freshmen averaged approximately five 
hours. All subjects were cooperative for 
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the most part, although the patients as a 
group were slower in beginning to write 
and less consistent in keeping at it, and 
therefore required more attention and 
encouragement. In no case, however, 
were topics suggested to the subjects or 
‘coaching’ resorted to in order to obtain 
the requisite length of sample. The total 
time elapsed during the securing of the 
samples was approximately two weeks 
for the patients and approximately one 
month for the freshmen (with the excep- 
tion of the four freshmen mentioned in 
the previous footnote). 

The 2800-word samples were typed ex- 
actly as they were written. Each sample 
was then divided into twenty-eight suc- 
cessive one-hundred-word segments by 
counting the first one hundred words, 
placing a mark, and then counting the 
second hundred words, etc. Each word 
was then tabulated on sheets so designed 
that each one-hundred-word segment 
could be tabulated separately.’ The pro- 
cedure followed in tabulating the data 
was as follows. 

After a sample had been _ typed, 
double-spaced, one of a pair of workers 
(much of the time one worker performed 
these tasks alone) placed numbers, one 
to one hundred, over the first one hun- 
dred words. These numbers, one over 
each word, were written very small. After 
the one hundredth word a small number 
one was written and encircled—to indi- 
cate the limit of the first one hundred 
words. The other worker, meantime, had 
written a letter of the alphabet in the 
upper-left hand corner of each of sev- 
eral tabulation sheets. The first word of 
the first one hundred words was noted 
and worker No. 1 looked all through the 


*A copy of the tabulation sheet is in the 
appendix of the manuscript copy of this report 
which is on file in the State University of Iowa 
Library, 
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one-hundred-word sample, counting the 
number of times the word appeared. 
Worker No. 2 wrote this word, followed 
(in parenthesis) by the part of speech it 
represented in the “Word” column on 
the tabulation form that carried the let- 
ter of the alphabet under which the 
word would be classified alphabetically. 
The number of times the word appeared 
in the first hundred words was noted in 
the column headed “1” on the tabula- 
tion form. A small check was placed over 
the number, which had previously been 
placed over the word, as each word was 
counted. 

After the first one hundred words had 
all been counted and tabulated, worker 
No. 1 counted off the next hundred 
words, numbering them from one to one 
hundred and placing an encircled 2 just 
after the last word of this second one- 
hundred-word section. Worker No. 2 
totaled the frequencies noted in the col- 
umn headed “1” on all of the tabulation 
forms used in order to check that the 
total was one hundred. The frequencies 
of the second one hundred words were 
noted in the column headed “2”. Only 
the words appearing in this second one- 
hundred-word segment that did not ap- 
pear in the first one hundred were writ- 
ten in the “word” column. This proce- 
dure was continued throughout the 2800- 
word sample. 

The following rules were used in de- 
termining what constituted a word: 

1. Each group of letters separated by spaces 
on both sides from adjacent groups of 
letters was counted as a word, even 
though it might be part of a place name, 
as in Des Moines (two words), an initial, 
as in James A. Brown (three words), or 
a neologism coined by a subject. 

2. Any number was counted as one word; 


for example, 125 was tabulated as one 
word. 


3. A hyphenated word was counted as one 
word, Webster’s New International Un- 


abridged Dictionary (23) being used as 
the authority as to whether or not a 
word should be hyphenated. 

4. Each time a word was used as a differ- 
ent part of speech it was counted as a 
different word. For example, mine as a 
noun and mine as a pronoun were 
tabulated as two different words. 

5. Common nouns and proper nouns hav- 
ing identical spellings were thrown to- 
gether. For examples, the two words 
Storm Lake were tabulated under the 
common nouns storm and lake. 

6. Contractions were divided into two 
words, for example, didn’t was changed 
to did not and tabulated as two words. 

. Abbreviations which stood for only one 
word were written out and tabulated as 
the complete word. Abbreviations which 
consisted of more than one unit, as for 
example M.D. and Ph.D., were tabu- 
lated as one word. 

8. Misspellings, when it was apparent that 
they were misspellings and not neo- 
logisms were corrected and tabulated as 
corrected. 


~I 


The part of speech was placed after a 
word as it was tabulated. Following is a 
list of the rules which were used in de- 
termining the part of speech represented 
by any given word. To be classified as: 


Nouns—all regularly known common and 
proper nouns and gerunds which the 
dictionary® recognizes as nouns. 

Pronouns—all personal and indefinite pro- 
noun forms, including pronominal ad- 
jective forms, such as my, our, your, 
their, etc. Also all demonstrative, rela- 
tive, and interrogative pronouns such 
as this, those, who, whom, where, etc. 

Verbals—simple verbs, participles plus aux- 
iliaries, gerunds and participles unless 
the dictionary recognizes them as nouns 
and adjectives, as the case may be. 

Adjectives—regular classification, and any 
verb form (i.e. participle) which the dic- 
tionary recognizes as an adjective. 

Adverbs—regular classification. 

Prepositions—regular classification. 

Conjunctions—regular classification. 

Interjections—exclamatory expressions, and 


* Webster’s New International Unabridged 
Dictionary (23). 
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slang expressions used interjectionally. 
Articles—a, an and the. 


The data on the tabulation sheets 
were then analyzed and will be presented 
in three different sections: (1) Type- 
Token Ratios (TTR’s), including both 
segmental TTR’s and overall TTR’s; 
(2) Grammatical Analysis; and (3) Type 
Frequencies. 


The Type-Token Ratio 

The type-token ratio® is a quantitative 
measure of language to which most at- 
tention has been given in the present 
study. Ihe number of types in a given 
language sample is the number of differ- 
ent words occurring in the sample, and 
the number of tokens is the total number 
of words in the sample. The type-token 
ratio, then, is computed by dividing the 
number of different words by the total 
number of words in the sample. Since it 
may be assumed, from the work of Car- 
roll (10), that the percentage of different 
words decreases as successive increments 
are added to a language sample, the 
number of tokens used in computing the 
type-token ratio must be kept constant 
in order to determine any variations 
within any given language sample, or in 
order to make the ratio comparable from 
one sample to another. 

In this study the computations of the 
‘TTTR’s have been (1) the overall TTR 
as computed for the entire sample of 
2800 words, and (2) the mean segmental 
TTR. As was stated previously, in this 
study each 2,800-word sample was di- 
vided into twenty-eight successive one- 
hundred-word segments. To secure the 
mean segmental TTR’s the TTR was 
computed for each one-hundred-word 
segment independently and these seg- 


mental TTR’s were averaged for each 


* This term was introduced by Johnson (15) 
and the ratio has been discussed by him. 


sample. This procedure makes it possible 
to compare samples of different magni- 
tudes since such segmental TTR’s are 
directly comparable as long as they rep- 
resent segments of equal size, and the 
means of such segmental TTR’s and 
mean segmental ‘IT R’s from the present 
study can be compared with those from 
any other study involving one-hundred- 
word segments, regardless of the num- 
ber of such segments in a given sample. 


Consideration of the TTR Scale 


The limits of the TTR are mathe- 
matically defined as greater than zero 
and equal to or less than one. As to the 
nature of the cumulative TTR curve, it 
may safely be stated that D (the number 
of different words, or types) is a complex 
function of N (the total number of 
words, or tokens, in the sample). The 
greater the base on which the TTR is 
computed the smaller the absolute value 
of the TTR will be for any one sample 
of any one individual.*° 

The question may arise as to the rela- 
tive value of the TTR unit at any given 
position on the scale from zero to one. 
This question is more obvious when it 
is considered whether the difference of 
one I'TR unit at one point on the scale 
is equal to a difference of one TTR unit 
at any other point on the scale. First of 
all, the operational character of the 
TTR unit is clear. The question here 
raised would appear to be significant, if 
ever, whenever interpretations might be 
drawn as to the relation of the TTR to 
some other variable. It is to be pointed 
out that in any segment of the TTR 
scale where the variability of the TTR’s 
for any given group of language samples 
is relatively large, a correspondingly 
larger absolute difference between any 


* The problems implied by these statements 
are treated in greater detail by Chotlos (11). 
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two I°TR’s would be required to satisfy 
the criteria of statistical significance, 
than would be required in any segment 
of the scale where the variability of 
TTR’s is relatively less. In this sense, 
then, the question becomes one of the 
relative difference with regard to varia- 
bility of TTR’s at different points along 
the scale, and insofar as there are differ- 
ences in such variability, it is to be ex- 
pected that there will be corresponding 
differences in the relative significance 
statistically to be ascribed to differences 
of the same absolute magnitude, de- 
pending upon the segment of the scale 
which they involve. However, safeguards 
against misinterpretations that might 
conceivably result from this fact are to 
be found in the statistical procedures to 
be used in treating the data that are to 
be interpreted; it is not the similarity of 
two differences with regard to their abso- 
lute magnitude but the similarity with 
regard to their relative magnitude as 
shown in their degree of statistical sig- 
nificance, that would govern any in- 
terpretation regarding them. A logical 
consideration of the TTR scale would 
indicate that a mean TTR value at 
either the upper or lower end of the 
scale should imply a lower ‘degree of 
variability among the individual TTR’s 
of which it is the mean than would a 
mean TTR value in the middle range 
of the scale. This is true because varia- 
tion from the mean in the direction of 
zero, in the case of a low mean TTR 
value, would obviously be limited in ex- 
tent, and any relatively large variations 
from the mean in the opposite direction 
would tend to raise the mean; the same 
type of consideration would hold with 
regard to a relatively high mean TTR 
value. It is obvious, on the other hand, 
that a mean TTR value approximating 
.50, for example, does not necessarily 


imply any such limited range of devia- 
tions of the individual TTR’s from the 
mean. 

In order to make a partial investiga- 
tion of the question under discussion 
Pearson product-moment correlations 
were run between the mean segmental 
T’TR’s and the standard deviations, sep- 
arately for the psychotic subjects and for 
the freshmen. This correlation for the 
psychotic subjects was —.og, and for the 
freshmen it was —.12. Neither of these 
values deviates significantly from zero. 
This may be interpreted to mean that 
for each of the groups the TTR’s fell 
within a segment of the scale within 
which there would appear to be no ap- 
preciable relation between the absolute 
magnitude of the TTR and its variabil- 
ity. However, the trend is in the direc- 
tion indicated by the above logical con- 
siderations, and it may be assumed that 
the low correlations obtained are to be 
accounted for in part, at least, by the 
fact that T'TR’s for each group fell 
within a relatively narrow range. 

In order to ascertain the degree of re- 
lation between the absolute value of the 
mean segmental TTR’s and their vari- 
ability when a larger number of meas- 
ures and a larger range of the scale were 
involved, a Pearson product-moment 
correlation was run between the mean 
segmental TTR’s and the standard de- 
viations for all subjects. The correlation 
obtained was —.58. The fact that this 
correlation coefficient is higher than 
either of the corresponding coefficients 
for the separate groups tends further to 
substantiate the above logical considera- 
tions. 


It is to be emphasized again, however, 
that the relationship implied by these 
coefficients of correlation and by the 
logical analysis of the scale are of no 
particular significance so far as the inter- 
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pretation of differences in TTR values 
is concerned, since the differences are to 
be interpreted with reference to their 
relative rather than their absolute mag- 
nitude. Misinterpretation would occur 
only if the indicated relationships were 
ignored; they are, of course, taken into 
account in the statistical procedures on 
the basis of which the significance of the 
differences between the TTR values is 
estimated. 


Grammatical A nalysis 


The grammatical analysis is concerned 
with ascertaining the proportion of the 
entire language sample, for each subject 
and for each group of subjects, that is 
represented by each of the parts of 
speech. Relationships between certain 
parts of speech have been computed in 
terms of ratios. 


Type Frequencies 

The section on type frequencies is con- 
cerned with an objective language meas- 
ure which expresses relative frequency 
of occurrence of each different word, or 
type. Of particular interest are those type 
frequencies which differentiate the writ- 
ten language of schizophrenic patients 
from that of freshmen. In order to select 
such types, if they exist, the one hun- 
dred most frequently used types were 
found for each group and comparisons 
of these were made. Particular attention 
was also given to certain types such as 
self-reference words and ‘allness’ terms, 
such as never, always, all, etc. 

Also, the proportionate vocabularies 
of the two groups were compared. The 
proportionate vocabulary is found by de- 
termining the number of types making 
up a certain proportion of the tokens in 
a given language sample. Finally, a word 
list was compiled which presents each 
type separately and shows the number 


of subjects in each group who used the 
word, and the type frequency for each of 
the two groups.?! 


IV. RESULTS 
Introduction to Results 


In order to facilitate the discussion of 
the results the following system of sym- 
bols has been The reader is 
asked to refer to this list for definitions 
of the symbols in terms of the operations 
to be performed in deriving the statis- 
tics which they represent. 

The data were analyzed to determine 
the characteristics of the type-token ra- 
tios for one-hundred-word segments. The 
following symbols will be used in dis- 
cussing the results of this section of the 
analysis. 


devised. 


D 
Let TTR=R= N where D is the number of differ- 


ent words (types) in a segment and N is the total 
number of words (tokens) in that segment. 


Let R,=segmental TTR where p is the subscript for 
any given one-hundred-word segment. 


Let Ri, Re, Rz,--- Rp,- ++ Res refer to segmental 
TTR’s for each one-hundred-word segment, one 
through twenty-eight. 


where D;, Do,--+- Dp,- ++ Dog are independently 
computed, the number of different words in any 
one segment not being influenced by any words in 
any other segment, and N,;=N2=--- 


N>= : 
Nos = 100. 


Let Rj represent the mean of Rj, Ro,--- Rp--- Ros 
for each individual subject. 
=R 
Ri=Ri+R2+ +--+ +Rpt+::: +in=—: 
2 


Let sj represent the standard deviation of the 
R;, Ro, - ++ Rp, + + + Res for each individual subject. 


/2Bo~Ri}* 


i= 4 


"This complete word list is contained in the 
appendix of the manuscript copy of this report 
on file in the State University of Iowa Library. 
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The data were analyzed in order to 
determine the characteristics of the seg- 
mental TTR for the group. The follow- 
ing symbols will be used in the discus- 
sion of the results of this part of the 
analysis. 


Let Rm represent the segmental TTR for the group 


a i 





Rn= where Rj is the mean TTR per subject 


n 


summed over all the group and n is the number of 
subjects in the group 


Let Sm represent the standard deviation of the dis- 
tribution of mean segmental TTR’s (Ri’s) for the 
group. 


,/= os Rm)? 
Sm = —_—_—__ —_ —_ —- e 


n 


~ Let S.E.m represent the standard error of the group 
mean segmental TTR(R,,). 


>» —— m)? 2s 
Let S.Em= 4/ i = ae 
n(n—1) Vn-I 





Let Mi represent the mean of the standard devia- 
tions for each of the n subjects in a group. 


Lsi 


Msi= 
n 


Let os represent the estimated variance of the 
standard deviations for the group 


est. o%,;=—_____- 


The data were analyzed to determine 
the characteristics of the TTR when it 
is computed by considering the entire 
sample as a whole. This TTR is called 
the overall TTR and the following sym- 
bols will be used in discussing the results 
of this section of the analysis. 


Let R’ represent overall TTR 


, 


R’= N’ where D’ is the number of different words 





(types) and N’ is the total number of words (tokens) 
in the entire sample. Computed independently for 
each subject. 


Let R’m represent the mean overall TRR for the 
group. 


Cr 
wn 
~~ 


of 
a 





where n is the number of subjects in the 
group. 


Let s’m represent the standard deviation of the 
overall TRR’s for the group. 


rae a 
" n 


Let S.E.’m represent the standard error of the group 
mean overall TRR. 


——, =(R’—R’‘n)? 
S.E.'m= /—— ’ 
n(n—1) 


Let o}%,, represent the estimated variance of the 
overall TRR’s for the group. 
=R’—R’'n 


aA~Tt 


oc? = 








I. TYPE-TOKEN RATIO 
Internal Consistency of Segmental 
TTR’s 

It was felt that it would be desirable 
to secure some measure or indication of! 
the internal consistency (i.e. how well a 
random half of the sample measures 
what the whole sample measures) of the 
2800-word samples for each subject. This 
was obtained by splitting the R,, R,, . . 
R,, .- - R,, for each subject at random 
into two sets of R’s. The mean for each 
random half was computed and the ¢- 
test for related measures was applied.’ 
It would be expected from such a test 
that if the internal consistency of the 
samples was high, the value of ¢ so de- 
rived would fail to be statistically signifi- 
cant. When this test was applied to the 
random sets of R’s for the patients the 
value of t was 1.82, and when applied to 
the random sets of R’s for the freshmen 


2 See Lindquist, E. F., Statistical Analysis in 
Educational Research, Houghton Mifflin Com- 
pany, Boston, 1940, p. 58. The procedure is that 
of finding the difference for each pair of R’s 
and for this distribution of differences, deter- 
mining whether or not the mean difference dif- 
fers significantly from zero. 
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the value of ¢t was .411. The values in 
both cases fall short of significance at the 
five per cent level of confidence with 
twenty-three degrees of freedom (d.f.). 


Variability in Segmental TT R’s 


We were interested in determining 
whether the schizophrenic patients were 


TABLE 1 


TTR’s for each subject ranked in descending 
order within each group 








Mean-Segmental TTR’s Overall TTR’s 





Patients Freshmen Patients Freshmen 





- 7450 
- 7404 
. 7386 
.7164 
. 7007 
-6975 
.6846 
-6757 
.6700 
.6700 
.6700 
. 6668 
.6657 
6618 
.6607 
.6582 
.6482 
.6436 
.6389 
.6264 
- 5993 
. 5700 
- 5346 
. 4600 


-7357 
-7354 
-7339 
- 73°97 
- 7293 
.7286 
-7279 
. 7261 
. 7236 
. 7236 
. 7200 
.7196 
-7143 
7118 
. 7104 
. 7082 
- 7957 
- 7954 
-6975 
.6946 
-6943 
.6932 
.6836 
.6708 


- 3932 
3854 
. 3618 
- 3596 
- 3497 
-3154 
.3150 
. 2961 
. 2946 
. 2821 
. 2789 
- 2779 
.2746 
.2725 
. 2639 
-2575 
. 2464 
. 2371 
. 2371 
.2279 
. 2121 


- 2121 


- 1943 
. 1850 


-4979 
- 3907 
. 3607 
3471 
- 3457 
- 3454 
- 3450 
- 3439 
.3411 
-3375 
- 3397 
- 3293 
. 3289 
. 3250 
. 3229 
.3218 
. 3104 
. 3089 
. 3086 
- 3014 
. 2971 
. 2921 
. 2879 
. 2689 





more variable than the freshmen, not 
only from subject to subject, but whether 
they also showed more variability from 
segement to segment than did the fresh- 
men. In order to determine this the s, 
for each subject and Ms, for each group 
were computed. The F ratio'® when com- 
puted as a ratio of the variance of the 
distribution of s,’s for schizophrenic pa- 
tients to the variance of the distribution 


* See Lindquist, op. cit., p. 60. F, the variance 
12 


1 
ratio, is defined as —~— in which o,” and o,” are 
2 
O2 
estimates of the true variances of the popula- 
tions sampled. 
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of s,’s for freshmen resulted in a value 
of F of 10.35 which, with twenty-three 
and twenty-three d.f., is significant at 
the one per cent point. 

In order to determine whether or not 
there was a difference in variability from 
segment to segment between the sexes, 
the F ratio was computed as a ratio of 
the variance of the distribution of s,’s 
for the female subjects to the variance 
of the distribution of s,’s for the male 
subjects. The value of F so obtained for 
the patients was 4.49 which with eleven 
and eleven d.f. is significant at the one 
per cent point, the male patients show- 
ing more variability than the female pa- 
tients. The value of F so obtained for 
the freshmen was 1.47 which with eleven 
and eleven d.f. fails to be significant at 
the five per cent point, the value of F 
required for significance at that point 
being 2.82. 


Means and Distributions of Mean Seg- 
mental TTR’S and Overall TTR’s 


Fhe R, RB, . «+ Ry... Ry Ee 
subject were averaged and a mean seg- 
mental R (R,) obtained for each indi- 
vidual.'* An overall R (R’) was also ob- 
tained for each subject by considering 
the 2800-word sample as a unit and di- 
viding the number of types in the entire 
sample by 2800. The R,’s and R’”s for 
each group are ranked in descending or- 
der in Table 1. An examination of this 
table reveals that there is some overlap- 
ping between the two groups on both the 
R,’s and R’’s. The R,’s for three patients 
were higher than the highest R,; among 
the freshmen; and the R,’s for eight pa- 
tients were higher than the lowest R; 
among the freshmen. The lowest R’ 


* Table 1 in Appendix A of the manuscript 
copy of this report on file in the State Univer- 
sity of Iowa Library presents the twenty-eight 
R’s for each subject. 
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TABLE 2 


Group means, standard error of means, and standard deviation for mean-segmental 
TTR’s and overall TTR’s 

















Mean-Segmental TTR Overall TRR 
Rm S.E.m Sw Re SE’ = Sn 
All Patients .6559 .01322 .06404 . 2801 .01180 .05625 
Female .6468 .02138 .06134 .2782 .O15§50 .05180 
Male .6651 .01608 .05385 . 2819 .01855 .06078 
All Freshmen -7135 .00358 .01753 . 3291 .00636 .03072 
Female -7179 .00392 .O1254 . 3350 .01060 .03548 
Male 7091 .00590 .01776 . 3232 .00710 .02365 





among the freshmen was higher than the 
R’’s for ten patients. Only one R’ among 
the freshmen was higher than all R’’s 
among the patients. 

Table 2 presents the mean of the R,’s 
for the group (R,,) and the mean of the 
R”s for the group (R’,,) with the stand- 
ard deviations (s,, for the R,; distribution 
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and s’,, for the R’ distribution) and the 
standard error of the means (S.E.,, for 
R,,, and S.E.’,, for R’,,) for each group 
for all patients, female patients, male 
patients, all freshmen, female freshmen, 
and male freshmen. 

The curves drawn from the frequency 
distributions of the twenty-four mean 
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Fic. 1. Cumulative frequency curves of mean segmental TTR’s for 24 schizophrenics and 24 
freshmen. Means are shown by vertical lines 
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Fic. 2, Cumulative frequency curves of overall IT R’s for 24 schizophrenics 
and 24 freshmen. Means are shown by vertical lines 


segmental TTR’s (R,’s) of the schizo- 
phrenic patients and those of the fresh- 
men are shown in Fig. 1, and the curves 
drawn from the distributions of the over- 
all ‘IT R’s (R’’s) for both groups are 
shown in Fig. 2. It is apparent from the 
curves in Fig. 1 that the range of R,’s for 
the patients is greater than that for the 
freshmen, the range for the patients be- 
ing .4600 to .7450 while that for the 
freshmen is .6708 to .7357, indicating 
more variability among.the patients. The 
range of the R’’s was also somewhat 
greater for the patients than for the 
freshmen, the values ranging from .1850 
to .3932 for the patients, and from .2689 
to .4079 for the freshmen. The R,, for 
the patients was .6559 while that for the 


freshmen was .7135; the R’,, for the pa- 
tients was .2801 and that for the fresh- 
men was .3291. 


Mean Segmental TTR’s: Group Dif- 
ferences 

The t-test was applied to test the sig- 
nificance of the difference between R,, 
for the patients and R,, for the fresh- 
men.'® The value obtained for t was 
4.204 which, with forty-six d.f., is sig- 
nificant at the one per cent level of con- 
fidence. Therefore, we would feel justi- 
fied in rejecting the hypothesis that these 
samples were drawn from populations 
whose means are equal. 

However, since one of the assumptions 


®See Lindquist, op. cit., p. 56-58. 
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underlying the t-test when used to test 
the significance of the difference between 
means of independent small samples is 
that the true variance of one sample 
must be equal (or approximately equal) 
to the true variance of the other sample, 
a test of the significance of the difference 
in variability was applied. 

The F ratio when computed as the 
ratio of the variance of the distribution 
of R,’s for patients to the variance of 
the distribution of R,’s for the freshmen 
resulted in a value of F of 13.34. which 
with twenty-three and twenty-three d.f. 
is significant at the one per cent point. 
The results of this test indicate that the 
variability of the patients as a group ex- 
ceeds the variability of the freshmen as 
a group to an extent which cannot be 
attributed to chance fluctuations in ran- 
dom sampling. Another way of stating 
this is that we are ‘practically certain’ 
that the samples are drawn from differ- 
ent populations and that our ‘best esti- 
mate’ of the true variance of the popula- 
tion from which the sample of schizo- 
phrenic patients was drawn is consid- 
erably greater than the corresponding 
‘best estimate’ of the true variance of the 
population from which the sample of 
freshmen was drawn. 

Although we have no way of knowing 
the true variance of the populations 
which have been sampled, there is some 
question as to the validity of applying ¢t 
to test the significance of the difference 
in means in view of the difference in 
variability of the two groups. There- 
fore, the analysis of the data was ex- 
tended to get a further indication of the 
significance of the difference between 
the means for the two groups which 
would not rest upon the assumption of 
homogeneity of variance. This was ac- 
complished by using ¢ to establish limit- 
ing values for each group outside of 
which any exact hypothesis as to the 
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value of the true mean may be rejected 
with a given degree of confidence. At 
the one per cent level of confidence the 
limiting values of the true mean for the 
patients were .6188-.6930, and for the 
freshmen they were .7034-.7236. Since 
there is no overlap in these ‘confidence 
intervals’ we may be practically certain 
that the difference between the R,,,’s for 
the patients and for the freshmen indi- 
cates a real difference between the two 
groups. 

The critical ratio of the difference be- 
tween the R,,’s for patients and for 
freshmen was 4.204. The probability 
that a C.R. of this magnitude for inde- 
pendently drawn samples from these two 
populations will be exceeded solely 
through errors in random sampling is 
.0001. This value of the critical ratio is 
larger than the criterion usually required 
for statistical significance. 


The Effect of Certain Variables on 
TTR 


In an attempt to determine how cer- 
tain variables, particularly within the 
schizophrenic group, influence the TTR, 
the schizophrenics were sub-divided into 
groups and the average TTR’s for these 
sub-groups compared. The fifteen pa- 
tients for whom intelligence test results 
were available were split into two groups 
on the basis of I.Q., an I.Q. of 100 repre- 
senting the dividing line. Seven patients 
had 1.Q. scores below 100, ranging from 
78 to g7, with an average of 87; eight 
patients had I.Q. scores above 100, rang- 
ing from 101 to 138, the average being 
109. The mean segmental TTR’s for 
the individuals within each group were 
averaged, resulting in an average mean 
segmental TTR of .6800 for the “above 
average I.Q.” group and .6586 for the 
“below average I.Q.” group. The t-test 
of the significance of this difference in 
the average mean segmental TTR’s for 








| i 
i | 
i 
! 

i | 























58 MARY BACHMAN MANN 


these groups resulted in a t value of .51 
which with thirteen d.f. is not statisti- 
cally significant. 

Level of educational attainment would 
appear to be a variable among the pa- 
tients which might influence the TTR’s. 
To get an indication of how this factor 
affects the I'T'R, the patients were sub- 
divided into three groups on the basis 
of level of educational attainment: ten 
patients who had college training; six 
patients who had graduated from high 
school but who had had no college 
training; and eight patients who had 
not graduated from high school. The 
average mean segmental TTR for each 
group was computed and a simple analy- 
sis of variance technique was used to 
determine whether the differences in 
means for the three groups are signifi- 
cant of real differences, or may be ex- 
plained away in terms of chance fluctua- 
tions in random sampling. The mean 
segmental TTR’s were .6462 for college 
graduates, .6876 for high school gradu- 
ates, and .6395, for the lowest educational 
group, or non-high school graduates. 
The ratio (F) of the estimate of the 
populations variance, based on the vari- 
ance of the group means, to the estimate 
of the population variance, based on 
variance within groups, resulted in an 
F value less than unity which obviously 
is not significant. 

Duration of illness might conceivably 
be an important variable influencing the 
TTR’s for the patients. Since duration 
of illness can at best be only roughly 
estimated, it was felt that the effect of 
duration of confinement in the hospital 
could be more reliably ascertained. Since 
the average length of confinement in the 
hospital was three years, the patients 
were sub-divided with this average as a 
criterion. The thirteen patients who had 
been confined in the hospital for a 


shorter period than three years had an 
average mean segmental TTR of .6579, 
while the patients who had been con- 
fined in the hospital three years or longer 
had an average TTR of .6536. The t-test 
of the significance of the difference be- 
tween these means is not statistically sig- 
nificant. 


Comparison of TTR’s Computed 
from Written and Spoken Language 

A study by Fairbanks (12) in which 
she compared mean segmental TTR’s, 
using one hundred tokens as the size of 
the segment, for spoken language sam- 
ples from schizophrenic patients and su- 
perior freshmen, yielded results com- 
parable to those obtained in this in- 
vestigation. These studies are highly 
similar in that the subjects used in both 
were drawn from the same populations 
and the procedures followed in tabulat- 
ing and analyzing the data were essen- 
tially the same, but there is one point of 
difference which warrants some consid- 
eration. In her study Fairbanks em- 
ployed an interview situation, involving 
the use of fourteen proverbs, the inter- 
view being recorded by means of an elec- 
tric dictaphone technique without the 
subject’s knowledge, and with instruc- 
tions to the subjects to continue talking 
about anything that they wished to after 
finishing the proverbs. In the present 
study the instructions to the subjects 
were to write “the story of your life’. It 
is doubtful that much importance should 
be attached to this difference in meth- 
ods of securing the data, inasmuch as 
the samples of language obtained were 
probably of sufficient length to compen- 
sate for such differences. 

In general, the main findings of Fair- 
banks as to the differences between the 
spoken language of schizophrenic pa- 
tients and freshmen students were in the 
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same direction as those reported here 
for written language, the patients show- 
ing more variability as a group than the 
freshmen, and the difference between 
the mean segmental T'TR’s for the two 
sroups being statistically significant. 
Of particular interest is the compari- 
son between spoken language and writ- 
ten language as indicated by these two 


Overall TT R’s 

In determining the differences in over- 
all ‘I°'T'R’s between the groups the steps 
in the analysis followed those presented 
for the differences in segmental T'TR’s 
for the groups. 

The R’,, for the patients as a group 
was .2801 while that for the freshmen 
was .3291. The t-test applied to test the 


TABLE 3 


The average mean segmental TTR’s (R,..) for each group and the range values within each group for 
written and spoken language of schizophrenic patients and freshmen students. 
The data for spoken language are from Fairbanks (12) 








Written Language 


Spoken Language 











Rum Range Ru Range 
Schizophrenic Patients .6559 .4600—.7450 . 5681 .4933-.6193 
Freshmen Students -7135 .6708—.7357 .6416 .6137—.6650 





studies. Table 3 presents the average 
mean segmental TTR’s for each group 
and the range values, taken from the 
mean segmental TTR’s for the individ- 
uals in each group, for written and 
spoken language of schizophrenic pa- 
tients and freshman students. It may be 
readily observed from this table that the 
mean TTR for both types of subjects 
‘runs considerably higher for written lan- 
guage than for spoken language. This 
difference might have been anticipated 
because of the fact that in producing 
written material the individual has op- 
portunity and ample time to alter and 
rearrange the words that he writes, which 
in many cases amounts to striving for 
variety or ‘diversity’ in the words used. 
Thus, this premeditated aspect of written 
language tends to obliterate the spon- 
taneity which is more characteristic of 
spoken language. 

It is interesting to note that the spoken 
language of freshmen is characterized by 
approximately the same mean segmental 
TTR value as is the written language of 
schizophrenics. 


significance of the difference between 
these R’m’s resulted in t = 3.65, which 
with forty-six d.f. is significant at the one 
per cent level of confidence. The results 
of this test indicate that the difference 
in R’,,’s for the patients and for the 
freshmen is a real difference. 

The F test of the significance of the 
difference between the variances of the 
distdribution of R”s for the two groups 
yielded a value of 3.35. The value re- 
quired for significance at the one per 
cent point with twenty-three and twenty- 
three d.f. is 2.70, While the obtained 
value of F is greater than that required 
for significance at the one per cent point, 
it is not much greater. 

The further test, which has been dis- 
cussed previously, of using ¢ to set limit- 
ing values of the true mean of each 
group was applied to the R’,,’s of pa- 
tients and of freshmen. The limiting 
values of the true mean for the patients 
at the one per cent level of confidence 
were .2370-.3132 while the limiting val- 
ues of the true mean for the freshmen 
were .3113-.3469. There is a slight over- 
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lap in the intervals for the patients and 
the freshmen at the one per cent level 
of confidence, the upper limit for the 
patients extending .0019 above the lower 
limit for the freshmen. The limiting 
values of the true mean for the patients 
at the two per cent level of confidence 
were .2506-.3096 while those for the 
freshmen were .3132-.3450. Thus we are 
able to say that at the two per cent level 
of confidence there is a true difference 
between R’,,’s for the patients and the 
freshmen. 

The critical ratio of the difference be- 
tween the R’,,’s for patients and for 
freshmen resulted in a value of 3.654. 
The probability that a C.R. of this mag- 
nitude for independently drawn samples 
from these two populations will be ex- 
ceeded solely through errors in random 
sampling is .ooo0g. This test again indi- 
cates that the difference in R’,,’s for the 
two groups is statistically significant. 


Sex Differences 

Since each group of twenty-four pa- 
tients and twenty-four freshmen con- 
sisted of twelve male and twelve female 
subjects, the data were analyzed to de- 
termine whether there were significant 
differences between the sexes within each 
group for the R,,’s and R’,’s. 

The F ratio when computed as the 
ratio of the variance of the distribution 
of R,’s for the female patients to the 
variance of the distribution of R,’s for 
male patients gave an F value of 1.729 
which, with eleven and eleven d.f., would 
be exceeded by chance in more than five 
per cent of similarly selected random 
samples. The results of this test give us 
no adequate basis for rejecting the 
hypothesis that these samples were drawn 
from equally variable populations. Like- 
wise, the t-test of the significance of the 
difference between the R,,’s for the fe- 


male patients and the male patients re- 
sulted in a value of t of .683 which, with 
twenty-two d.f., is clearly not significant, 
since a value of this magnitude can be 
expected to occure by chance more than 
fifty per cent of the time in similarly 
selected random samples. 

The F ratio when computed as the 
ratio of the variance of the distribution 
of R,’s for female freshmen to the vari- 
ance of the distribution of R,’s for male 
freshmen gave an F value of 2.661 which, 
with eleven and eleven d.f., is not statisti- 
cally significant. This value can be ac- 
counted for by chance fluctuations in 
random sampling and we are therefore 
not justified in rejecting the hypothesis 
that the samples were drawn from 
equally variable populations. Similarly, 
the t-test of the significance of the dif- 
ference between R,,’s for female fresh- 
men and male freshmen resulted in a 

= 1.24 which, with twenty-two d.f., is 
not statistically significant since a value 
of ¢ of this magnitude can be expected 
to occur by chance between twenty and 
thirty per cent of the time. The results 
of this analysis give us no adequate 
basis for assuming any difference in R,,’s 
between male and female patients or 
between male and female freshmen. 

The ratio F when computed as a ratio 
of the variance of the distribution of 
R’’s for female patients and the variance 
of the distribution of R’’s for male pa- 
tients resulted in an F of 1.431 which, 
with eleven and eleven d.f., would occur 
by chance more than five per cent of the 
time in similarly selected random sam- 
ples. The t-test when applied to the dif- 
ference in R’,,’s for female and male pa- 
tients resulted in ¢ = .153 which, with 
the twenty-two d.f., would occur by 
chance more than eighty per cent of the 
time. The same tests when applied to the 
distributions of R’’s for female and male 
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freshmen resulted in values of F = 2.231 
which, with eleven and eleven d.f., is not 
statistically significant, and when applied 
to the difference in R’,,’s for female and 
male freshmen the resulting t = .g22 
which, with twenty-two d.f., would be 
expected to occur by chance between 
thirty and forty per cent of the time in 


2. GRAMMATICAL ANALYSIS 


Distributions and Group Differences 


The data were analyzed to determine 
the relative frequency of usage of each 
of the eight conventional parts of speech, 
plus articles (which were treated sepa- 
rately from other adjectives). Table 4 


TABLE 4 


Relative frequency of usage of different parts of speech expressed as percentage of the total number of 
words used by the group (67,200), with standard deviations of the distributions of five main 
categories. The range values are from individual samples* 








Schizophrenic Patients 


Freshmen Students 








Percentages S.D. Range Values Percentages S.D. Range Values 
Nouns 24.27 3.98 17.43-33 .68 22.15 2.26 17.86-25.57 
Pronouns 13.12 3.78 4.79-20.25 14.57 I.50 11.68-17.07 
Verbs 19.82 2.30 15 .86—23.93 18.71 1.60 16. 18-22. 36 
Adverbs 7.70 1.71 3.68-10.57 8.34 1.05 6.00-10.79 
Adjectives 8.33 2.56 4.68-16.00 9.45 ae iy 6.89—-10.96 
Conjunctions 7.23 3-75- 9.46 6.55 4.32- 8.29 
Prepositions 12.33 7.75-16.57 12.35 10.46-14.43 | 
Interjections 0.07 0.04- 0.86 0.05 ©.00- 0.21 
Articles 7.15 4.96-I11.00 7.83 §.2I-10.11 





* Table 1 in Appendix B of the manuscript copy of this report on file at the State University of 
lowa Library contains the percentage of usage of each part of speech for each individual. 


such samples. Our conclusion again is 
that the differences in R’’s between the 
sexes for the two groups are not Statisti- 
cally significant and we are not justified 
in rejecting the hypothesis that the sam- 
ples consisting of females and males, re- 
spectively, in each group were drawn 
from the same populations. 


Correlation Between Mean Segmental 
and Overall TTR’s 


The Pearson product-moment correla- 
tion coefficient between the R,, and R’,, 
was .62 for patients and .62 for freshmen. 
For all forty-eight subjects r = .71. The 
fact that the r for all subjects is greater 
than the r for either the patient or the 
freshmen group may be due to the bi- 
modality of the distribution for all sub- 
jects, or to the discrepancy in variability 
between patients and freshmen, or to an 
interaction of these two factors. 


presents these frequencies, expressed as 
percentages of the total number of words 
(67,200) used by each group, separately 
for schizophrenic patients and for fresh- 
men. The standard deviations of the dis- 
tributions of the five main categories, 
and range values for each category taken 
from individual samples are also in- 
cluded in the table. 

The statistical significance of the dif- 
ference between the groups was tested 
for adjectives, adverbs, nouns, pronouns, 
and verbs. Of particular interest were 
the differences in percentages of the to- 
tal number of words represented by each 
of these grammatical categories, and dif- 
ferences in variability of usage of these 
parts of speech. 

The t-test was applied to test the sig- 
nificance of the difference between pa- 
tients and freshmen in percentages for 
certain parts of speech. The values of ¢ 
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TABLE 5 
Values of ¢ and F obtained from testing signi- 
ficance of the difference in usage of certain gram- 
matical categories based on percentage of total 


sample between schizophrenic patients and fresh- 
men 











Values of ¢t Values of F 





Adjectives 1.88 4-78 
Adverbs 1.54 2.68 
Nouns 2.22 3.11 
Pronouns 3.97 6.38 
Verbs 1.86 2.07 
n=24 





With forty-six d.f. the values of ¢ required for 
significance are: at the one per cent level of 
confidence ¢= 2.69; at the five per cent level of 
confidence ¢=2.01. With twenty-three and 
twenty-three d.f. the values of F required for 
significance are: at the one per cent point 
F = 2.72; at the five per cent point F= 2.01 


obtained for the categories tested are pre- 
sented in Table 5. The only ¢t value 
which might possibly be regarded as 
Statistically significant is that obtained 
for the difference in percentage of nouns. 
This ¢ is significant at the five per cent 
level of confidence. The differences be- 
tween schizophrenic patients and fresh- 
men in percentages for adjectives, ad- 
verbs, pronouns, and verbs may, there- 
fore, be attributed to chance fluctuations 
in random sampling. 

The F ratio when computed as the 
ratio of the variance of the distribution 


of percentages (based on total words per 
sample) for each grammatical category, 
used by the patients, to the variance of 
the distribution of percentages of the 
same category used by the freshmen, re- 
sulted in the values of F presented in 
Table 5. Each F value was statistically 
significant, the F values obtained for 
adjectives, nouns, and pronouns being 
significant at the one per cent point 
while those for adverbs and verbs were 
significant at the five per cent point. 
We may conclude that the variability of 
the patients as a group exceeded that of 
the freshmen as a group in relative fre- 
quency of usage of five grammatical cate- 
gories, by an amount which cannot be 
attributed to chance fluctuations in ran- 
dom sampling. 


Sex Differences 


Table 6 presents the relative frequency 
of usage of different parts of speech ex- 
pressed as percentage of the total number 
of words used by each sex (33,600) in 
each group, with the standard deviations 
of the distributions of the five main cate- 
gories. 

The t-test was used to test the signifi- 
cance of the differences between males 
and females in each group in relative 


TABLE 6 


Relative frequency of usage of different parts of speech expressed as percentage of the total number of 
words used by each sex (33,600), with standard deviations of the five main categories 








Schizophrenic Patients 





Freshmen Students 





Female Male 


Female Male 





Percentages S.D. 


Percentages S.D. 


Percentages S.D. Percentages S.D. 





Nouns 23.73 4.61 24.80 
Pronouns 13.99 4-39 12.26 
Verbs 20.22 2.59 19.39 
Adverbs 7.93 1.87 7.47 
Adjectives 8.43 2.02 8.24 
Conjunctions 6.88 7.58 
Prepositions 12.00 12.65 
Interjections 0.09 0.04 
Articles 6.73 7.57 


wre WwW 


.20 22.77 2.18 21.53 2.15 
.85 14.27 1.71 14.87 1.81 
-99 18.47 1.80 18.95 1.33 
-53 8.09 0.76 8.59 1.30 
.03 9.36 0.86 9.53 1.48 

6.909 6.12 

12.26 12.44 

0.05 0.05 

7-74 7-92 
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TABLE 7 


Values of ¢ and F obtained from testing significance of the difference, in usage of grammatical cate- 
gories based on percentage of total sample between the sexes within each group 








Schizophrenic Patients 


Freshmen Students 











Values of ¢ Values of F Values of ¢ Values of F 
Adjectives -174 2.24 (Males)* - 333 2.93 (Males) 
Adverbs .638 1.53 (Females) 1.088 2.91 (Males) 
Nouns .633 2.07 (Females) 1.333 1.03 (Females) 
Pronouns 1.094 2.37 (Females) .90O7 1.73 (Females) 
Verbs . 846 1.69 (Females) . 706 1.85 (Females) 





With twenty-two d.f. the values of ¢ required for significance are: at the one per cent level of con- 
fidence t= 2.819; at the five per cent level of confidence t = 2.074. 
With eleven and eleven d.f. the values of F required for significance are: at the one per cent point 


F = 4.46; at the five per cent point F = 2.82. 


* The sex which was more variable in each case. 


frequency of usage of certain parts of 
speech, ‘The values of t, presented in ‘Ta- 
ble 7, are not statistically significant for 
any of the grammatical categories tested 
either for schizophrenic patients or for 


freshmen. 


The F ratio when computed as the 
ratio of the variance of the distribution 
of percentages for each grammatical cate- 
gory (based on total words per sample) 


for female subjects, to the variance of 
the distribution of percentages for the 
same category for male subjects, resulted 
in values shown in Table 7, for patients 
and for freshmen. The F values obtained 
for adjectives and adverbs as between 
male and female freshmen exceed the 
value of F required for significance at 
the five per cent point. In each of these 
two categories the male freshmen were 


TABLE 8 

Comparison of the relative frequency of usage of parts of speech in written and in spoken language 
expressed as percentage of the total number of words used by the groups, 67,200 in the case of 
written and 30,000 in the case of spoken language. Data for spoken language from Fairbanks (12) 








Schizophrenic Patients 








Spoken Written 
% Range % Range 
Nouns 13.04 10.40-16.63 24.27 17.43-33.68 
Pronouns 22.68 19.33-24.75 13.12 4-79-20.25 
Verbs 26.28 24.27-30.47 19.81 15 .86—23.93 
Adverbs 11.54 7.00-17.97 7.70 3.68-10.57 
Adjectives 5-37 3-77- 7.10 8.33 4.68-16.00 
Conjunctions 6.55 4.10—- 8.77 7.23 3.75- 9.46 
Prepositions 7.48 4.30-10.00 12.33 7.75-16.57 
Interjections 2.64 0.53- 4-43 0.07 0.04- 0.86 
Articles 4.48 2.53- 6.87 7.15 4.96-11.00 
Freshman Students 
Nouns 15.39 12.67-18.53 22.15 17.86—25.57 
Pronouns 17.96 14.40-20.40 14.57 11.68-17.07 
Verbs 22.05 20.50-24.47 18.71 16. 18-22. 36 
Adverbs 10.16 8.87-11.20 8.34 6.00-10.79 
Adjectives 6.69 5-67— 7.87 9.45 6.89—-10.96 
Conjunctions 8.83 7.33-11.40 6.55 4.32- 8.29 
Prepositions 10.00 8.80-11.00 12.35 10.46-14.43 
Interjections 1.26 ©.47—- 2.00 0.05 ©.00—- 0.21 
Articles 6.79 5.27-— 9.07 7.83 §.2I-10.11 
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TABLE 9 
Rank order of increase in relative frequency of 
usage of parts of speech, expressed as percentage 
of the total number of words used by the group 
in written over spoken and spoken over written 
language. Schizophrenic patients and freshmen 


students. Data for spoken language from Fair- 
banks (12). 





Rank Order of Increase (Written over Spoken) 











Schizophrenic Freshman 

Patients % Students % 
Nouns 86.1 Nouns 43-9 
Prepositions 64.8 Adjectives 41.3 
Adjectives 55.1 Prepositions 23.5 
Articles 59.6 Articles 15.3 
Conjunctions 10.4 


Rank Order of Increase (Spoken over Written) 








Schizophrenic % Freshman % 
Patients Students 
Pronouns 72.9 Verbs 27.7 
Verbs 32.7 Pronouns 23.3 
Adverbs 49.8 Conjunctions 34.8 
Interjections 3671.4 Adverbs 21.8 
Interjections 2420.0 





more variable than the female freshmen. 


Comparison of Written and Spoken 
Language 


Table 8 presents a comparison of the 
relative frequency of usage of parts of 
speech in written language with that in 
spoken language, the latter data being 
taken from the above mentioned study 
by Fairbanks (12) concerned with the 
spoken language of schizophrenic pa- 
tients and freshman students. This com- 
parison is justified by the fact that the 
data presented from Fairbanks’ study 
were from samples drawn from the same 
two general types of subjects and were 
analyzed in essentially the same manner 
as were the data presented in this study. 
This latter consideration is of great im- 
portance in view of the fact that results 
in word count studies and grammatical 
usage analyses depend to a large extent 
upon the rules followed in determining 
what constitutes a word and the rules 
used in classifying words as to the parts 
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of speech represented by them. An ex- 
amination of Table 8 reveals several dif- 
ferences in the relative frequency of 
usage of parts of speech in the spoken 
and written language of schizophrenic 
patients and freshman students, respec- 
tively. These differences are summarized 
in Table g by showing the rank order 
of increase in usage of the various parts 
of speech in written over spoken and 
spoken over written language for each of 
the two groups. 

There is a marked increase in per- 
centage of nouns, adjectives, preposi- 
tions, and articles for both schizophrenics 
and freshmen, and in conjunctions for 
schizophrenics, in written language over 
spoken language. For both groups the 
amount of written over 
spoken language is greatest for the nouns, 
the patients showing 86.1 per cent in- 


increase in 


crease and the freshmen 43.9 per cent 
increase in nouns used in written over 
spoken language. 

There is an increase in percentage of 
pronouns, verbs, adverbs, and interjec- 
tions for both groups, and in conjunc- 
tions for the freshmen, in spoken over 
written language. The largest amount of 
increase in spoken over written language 
was 72.9 per cent in the pronouns for the 
patients and 27.7 per cent increase in 
verbs for the freshmen. (The increase for 
interjections, for both groups, was so 
great as to mean for all practical pur- 
poses that interjections are used only in 
spoken language.) The parts of speech 
for which there was increase in written 
over spoken language and increase in 
spoken over written language were the 
same for the two groups wtih the excep- 
tion of conjunctions, which showed a 
slight increase in written over spoken 
language for the patients, and an in- 
crease in spoken over written language 
for the freshmen. 
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Inter-relationships Among Parts of 
Speech 

Of the relationships between certain 
parts of speech, the adjective-verb quo- 
tient (Ava) is of perhaps the greatest in- 
terest, since it, or a variation of it, has 
been used by other investigators. Buse- 
mann (4), as reported by Boder (3), re- 
corded in shorthand a number of stories 
told by children of different ages and 
found a marked fluctuation of the rela- 
tionship between ‘qualitative’ and ‘ac- 
tive’ (dynamic) expressions. In the cate- 
gory of qualitative expressions he in- 
cluded not only adjectives, but also 
nouns and participles of verbs, when 
used as attributes to any other nouns; in 
the category of active expressions he in- 
cluded all verbs except the auxiliary. 
By dividing the number of verbs by the 
number of qualitative expressions he ob- 
tained a measure which he called the 
Action quotient (Aq.) of style. Busemann 
found that a rhythmical increase and 
decrease of the Aq. occurs with increase 
in age, which he believes to correspond 
to alleged rhythmical changes of emo- 
tional stability during childhood, ado- 
lescence, and youth. Furthermore, ac- 
cording to Busemann’s theory, these 
rhythmical variations continue through- 
out the whole lifetime and reflect rhyth- 
mical variations of emotional stability 
and creative power. 

Rorschach (19), again as reported by 
Boder (3), in classifying the interpreta- 
tions given by subjects to a series of ink 
blots, calculated the ratio between dif- 
ferent types of descriptions made. He 
found that the predominance of kinaes- 
thetic description (verbs) indicates mod- 
erate, sluggish motility, introversion, and 
little adaptability to reality, while the 
predominance of color descriptions 
(qualitatives) reflects the excited, but 
alert, exact, and rapid motility, extra- 


version, and better adjustment to reality. 

Stimulated by the suggestions made in 
these studies, Boder (3) set out to find 
whether there exist gross differences of 
adjective-verb ratios corresponding to 
differences in subject matter of various 
classes of writing. He inverted the pro- 
cedure of Busemann, however, and took 
the adjective as the numerator in order to 
obtain a measure which might (if Buse- 
mann is right) correlate positively with 
desirable traits. The ratio he used indi- 
cates the number of adjectives per one 
hundred verbs and is designated in pure- 
ly grammatical (as opposed to Buse- 
mann’s behavioral ‘action quotient’) 
terms as the Adjective-Verb Quotient 
(Ava). He found that for each of the kinds 
of writings studied, 1.e., plays, legal stat- 
utes, fiction, and scientific monographs, 
the distribution of Ava.’s shows sufficiently 
large differences to prove that as a rule 
the Ava, varies with the subject matter of 
the text. The Adjective-Verb Quotients 
reported in the present study are fairly 
comparable to the quotients reported by 
Boder, although the special rules fol- 
lowed by him in the word count analyses 
were somewhat different from the ones 
followed in the present analysis. The 
main differences were that in his study 
only attributive adjectives were counted, 
i.e., Only adjectives placed before the 
noun; quantitative and ordinal numerals 
were not counted; no forms of have and 
be were counted, nor were could, should, 
and would. Inasmuch as the rules fol- 
lowed in the present study differed from 
those of Boder in such a way as to in- 
crease both the number of adjectives and 
the number of verbs, we might expect 
the ratios to remain fairly comparable as 
between the two studies. 

Table 10 presents the Ava’s for all 
scizophrenic patients and all freshman 
students ranked in descending order. 
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TABLE 10 
Adjective-verb quotients for schizophrenic 
patients and freshman students for written 
language, ranked in descending order for each 
group 








Adjective-Verb Quotients 
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ten and spoken language, together with 
the mean quotients for adjectives to 
nouns, and adverbs to verbs for both 
groups for written and spoken language. 
Although the values for both of the latter 























Schizophrenics Freshmen quotients were larger for the freshmen 

ee 66 than for the patients, indicating the use 
92 64 of more adjectives per noun, and more 
.58 .62 : . 
5 6 adverbs per verb, these quotients did not 
-53 -02 
-53 -60 appear to be as differentiating as between 
-51 5 . . 
3 38 freshmen students and schizophrenic pa- 
-49 -58 tients as did the adjective-verb quotients. 
-42 5 . 
aa — The t-test was used to test the sig- 
-41 -54 nificance of the difference in mean Ava.’s 
oa e- derived from written language for pa- 
-39 -53 suas P 

. 36 52 tients and freshmen, resulting in a value 

- 35 5° : : : ; 

4 34 48 of 1.93 which, with forty-six d.f., is al- 

3 -33 = most significant at the five per cent level, 
20 4 ae ‘ 

; 30 42 the value needed for significance being 

te -30 “48 1.95. 

se -30 +37 95 ° 

F .29 35 Table 12 shows the comparison of 

1 § fe rs the mean Ava.’s for schizophrenic patients 

ey i and freshmen students for both written 

a | and spoken language, together with the 

of With the exception of two patients average Ava.’s obtained by Boder for each 

Y 4 whose Ava.’s were strikingly high, the of four different types of style of writing. 

Me 4 Ava.’s for six freshmen were higher than This table reveals that the mean Ava. 

® those of the patients, and the Ava.’s for _for the spcken language of schizophrenic 

“ee | nine patients were lower than the lowest patients falls slightly below that of 

T one for the freshmen. Table 11 presents Boder’s ‘normative’ style, while the mean 

4 the mean Ava.’s for both groups for writ- Ava. for freshman students on spoken 

s TABLE 11 

e Relationships between certain parts of speech expressed as ratios for each group. The ratios for spoken 

nS language were computed from Fairbanks’ data (12) 
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TABLE 12 


Comparison of the A,,’s obtained from written 
and spoken language of schizophrenic patients 
and freshmen students with those obtained by 
Boder 











Obtained 
Values of Ava. 
Schizophrenics, Written 43 
Freshmen, Written 51 
Schizophrenics, Spoken .20 
Freshmen, Spoken .29 
Boder’s Data: 
Conversational (drama) II 
Normative (legal statutes) .20 
Narrative (fiction) 35 
Descriptive (science) .76 





material falls midway between Boder’s 
‘normative’ and ‘narrative’ styles. The 
Ava.’S computed from written language 
samples are considerably higher than 
those computed from spoken language 
for both schizophrenic patients and for 
freshman students. The mean Ava. for 
schizophrenic patients on written ma- 
terial falls somewhat above that for 
Boder’s ‘narrative’ type, while the mean 
Ava. for freshmen on written material 
falls about midway between the Ava.’s 
for Boder’s ‘narrative’ and ‘descriptive’ 
types. The differences between written 
as opposed to spoken language for both 
groups correspond to the findings of 
Boder. He suggests that this may be ex- 
plained by the fact that 


“the time of writing is under the author's 
control; so that he can pay more attention 
to the style and choose the proper expres- 
sions. He has the possibility of rereading his 
material and inserting adjectives where 
found necessary, thus converting his material 
into a product of repeated and premeditated 
activity, lacking the spontaneity and speed 
which characterize the dialogue.” (3) 


3. TYPE FREQUENCIES 


Table 13 presents a list of the hun- 
dred most frequently used words for the 
schizophrenic patients and the freshmen 
students, respectively. The list for the 


freshmen has those words common to 
both lists arranged in order of frequency, 
while the list for the schizophrenics has 
the words corresponding to those of the 
freshmen arranged in order of sequence 
regardless of frequency. The seventeen 
words in each of the two groups not 
common to both lists are arranged at the 
bottom of the table in order of fre- 
quency. When this list of one hundred 
most frequently used words in written 
language of these two groups is com- 
pared with that reported by Fairbanks 
(12) for spoken language, we find that 
sixty-nine of the hundred are common 
to both lists for freshmen and sixty-four 
of the hundred are common to both 
lists for schizophrenic patients. 

Fairbanks reported some striking dif- 
ferences in the frequencies with which 
certain types occurred in the spoken lan- 
guage of schizophrenics and freshmen. 
She found, for example, that schizo- 
phrenics used not almost twice as many 
times as did the freshmen, and that no 
and never occurred in the schizophrenic 
list while not was the only negative word 
that occurred among the one hundred 
words most frequently used by the fresh- 
men. An examination of the words in 
Table 13 shows that these group differ- 
ences are not found in the written lan- 
guage. Fairbanks also reported that very 
was used three times more often by 
freshmen than by schizophrenic patients, 
while in the present study the patients 
used very almost three times more often 
than did the freshmen. 

Table 14 shows the relative frequency 
of occurrence of first person singular 
pronouns (I, my, mine, me, myself), first 
person plural pronouns (we, our, ours, 
us, ourselves), second person pronouns, 
singular and plural (you, your, yours, 
yourself, thee, thou), and third person 
pronouns, singular and plural (he, his, 
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to both lists are arranged in descending rank order according to frequency of usage for freshmen. 
The remaining 17 words not common to both lists are arranged in order of frequency 


Oo cx~)] 


wn 


1 


Vv 


ge 


w 


6. 


Om & Ww hd KH 


Orc 


Word 


the 
I 
and 
to 
was 
my 
in 
of 
a 
it 

we 

had 

not 

that 
with 

at 

for 

have 

on 

were 

but 

that 
school (s) 
this 

time 

is 

which 
when 
would 
she 

our 

did 

an 

from 

he 

her 

by 

or 

year 
mother 
as 

they 
first 

one 

us 

do 
about 
them 

so 

out 

who 

his 

as 

all 

one 

life 
years 





| 
| 
| 
| 
| 
| 





Freshmen 





for the two groups at the end of the Table 


Schizophrenics 


List of 100 words most frequently used by schizophrenics and freshmen. The first 83 words common 











>, 

Soccch Freq. Word 
art. 3354 the 
pro. 2778 I 
conj. 2350 and 
prep. 1805 to 
verb 1468 was 
pro. 1346 my 
prep. 1328 in 
prep. 1162 of 
art. 844 a 
pro. 672 it 
pro. 646 we 
verb 603 had 
adv. 552 not 
pro. 442 that 
prep. 440 with 
prep. 429 at 
prep. 428 for 
verb 421 have 
prep. 400 on 
verb 399 were 
conj. 395 but 
conj. 387 that 
noun 371 school (s) 
pro. 327 this 
noun 280 time 
verb 269 is 
pro. 269 which 
con}. 252 when 
verb 245 would 
pro. 239 she 
pro. 228 our 
verb 221 did 
art. 212 an 
prep. 211 from 
pro. 210 he 
pro. 198 her 
prep. 196 by 
con}. 192 or 
noun 189 year 
noun 182 mother 
conj. 181 as 
pro. 181 they 
adj. 167 first 
adj. 165 one 
pro. 163 us 
verb 162 do 
prep. 159 about 
pro. 159 them 
adv. 156 sO 
adv. 155 out 
pro. 151 who 
pro. 149 his 
adv. 142 as 
noun 140 all 
pro. 139 one 
noun 134 life 
noun 134 years 





Part of F 
Speech —— 
art. 3052 
pro. 2662 
conj. 2950 
prep. 2093 
verb 1069 
pro. 859 
prep. 1054 
prep. 16041 
art. 847 
pro. 507 
pro. 795 
verb 646 
adv. 468 
pro. 430 
pro. 457 
prep. 417 
prep. 139 
verb 416 
prep. 320 
verb 278 
conj. 215 
con}. 170 
noun 288 
pro. 236 
noun 287 
verb 585 
pro. 173 
conj. 297 
verb 350 
pro. 180 
pro. 150 
verb 184 
art. 158 
prep. 264 
pro. 298 
pro. 144 
prep. 192 
conj. 241 
noun II! 
noun 107 
conj. 319 
pro. 243 
adj. 102 
adj. 174 
pro. 122 
verb 210 
prep. 169 
pro. 163 
adv. 119 
adv. 194 
pro. 86 
pro. 132 
adv. 153 
noun 87 
pro. III 
noun 123 
noun 130 
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TABLE 13 (Continued) 














Freshmen Schizophrenics 
Word ec Freq. Word oie Freq. 
58. two adj. 131 two adj. 138 
59. then adv. 130 then adv. 186 
60. up adv. 129 up adv. 99 
61. could verb 128 could verb III 
62. went verb 124 went verb 290 
63. remember verb 120 remember verb 118 
64. too adv. 114 too adv. 178 
65. other adj. 113 other adj. 88 
66. are verb 112 are verb 224 
67. some adj. 112 some adj. 137 
68. all adv. III all adv. 87 
69. always adv. 110 always adv. 94 
70. good adj. 103 good adj. 152 
71. am verb 100 am verb 95 
72. so conj. 100 so conj. 139 
73. after prep. 97 after prep. 110 
74. there adv. 95 there adv. 353 
75. day (D) noun 96 day (D) noun 126 
76. very adv. 93 very adv. 252 
77. what pro. 93 what pro. 96 
78. if conj. 92 if conj. 109 
79. just adv. g2 just adv. 86 
80. go verb QI go verb 159 
81. took verb go took verb IOI 
82. quite adv. 86 quite adv. go 
83. get verb 85 get verb 102 
84. home noun 172 house noun 181 
85. high (H) adj. 169 young adj. 152 
86. little adj. 104 got verb 150 
87. never adv. 97 work noun 134 
88. teacher noun 07 also adv. 128 
89. into prep. 93 used verb 126 
90. more adv. Q2 Iowa noun 125 
QI. any adj. 90 people noun 120 
92. class noun 90 can verb 113 
93. things noun 90 been verb 112 
94. only adv. 85 father noun 105 
gs. still adv. 85 him pro. 99 
96. came verb 84 city (C) noun 90 
97. made verb 82 will verb 93 
98. than conj. 81 while conj. 82 
99. town noun 81 know verb 82 
100. great adj. 80 like verb 81 





TABLE 14 


Relative frequency of usage of the different personal pronouns in spoken and written language 
expressed as percentage of the total number of words used by each group, 30,000 for each group 
on spoken material, and 67,200 for each group on written material. Data for 


spoken language from Fairbanks (12) 








Spoken 


Written 





Schizophrenics Freshmen 





Schizophrenics Freshmen 





First person singular 

First person plural 

Second person singular and plural 
Third person singular and plural 


7 


10.42 


-34 
1.44 
5-52 


% 
3.69 
1.05 


2.14 
6.41 


7% To 
5-92 7-95 
1.59 1.57 
-16 .06 
2.81 3-13 

















him, himself, she, her, hers, herself, it, 
its, itself, they, their, theirs, them) in the 
written language of schizophrenic pa- 
tients and freshman students and Fair- 
banks’ spoken language. It is apparent 
from this tabulation of the data that her 
findings in regard to differences between 
the groups in spoken language were not 
substantiated with regard to written 
language. 


Proportionate Vocabulary 


From her spoken language data Fair- 
banks found that the schizophrenics used 
only thirty-three types to make up fifty 
per cent of the total number of tokens, 
while the freshman group used forty- 
Six types to arrive at the same percentage. 
In the present study of written language, 
the schizophrenics used ninety-five types 
to make up fifty per cent of the total 
number of tokens, while the freshman 
group used ninety-six types to make up 
the same percentage. For both groups 
ten types make up slightly over twenty- 
five per cent of the tokens in the written 
language. In connection with these com- 
parisons of proportionate vocabulary of 
written and spoken language it should 
be pointed out that the number of tokens 
used by each group for the written Jan- 
guage data was 67,200 while for the 
spoken language data the number of 
tokens for the schizophrenics was 29,800 
and for the freshmen it was 30,000. 

By dividing the number of types mak- 
ing up fifty per cent of tokens by the to- 
tal number of tokens in each case the 
following percentages were obtained: for 
written language, .14 for both freshmen 
and for patients, and for spoken lan- 
guage, .15 and .11 for freshmen and for 
schizophrenics, respectively. 

The patients used fifty-seven words 
which appeared to be privately coined 
words or neologisms while the freshmen 





70 MARY BACHMAN MANN 


used only five words which might be 
considered neologisms.** 


V. SUMMARY AND CONCLUSIONS 


This study is concerned primarily with 
the specific problem of determining 
whether and in what respects ‘adequate’ 
and ‘inadequate’ language might be dif- 
ferentiated quantitatively. “Twenty-four 
schizophrenic patients, twelve male and 
twelve female, were selected to represent 
a group presenting ‘inadequate’ lan- 
guage and twenty-four superior univer- 
sity freshmen, twelve male and twelve 
female, were selected to represent a 
group presenting relatively ‘adequate’ 
language. 

A 2800-word written language sample 
was obtained from each of the subjects 
under as uniform conditions as possible, 
the instructions to the subjects being to 
“write a story of your life.” Each sample 
thus obtained was divided into twenty- 
eight successive one-hundred-word seg- 
ments and each word, together with the 
part of speech it represented, was tabu- 
lated on sheets so designed that each one- 
hundred-word segment was _ recorded 
separately. Three types of analysis of the 
data were made: (1) the type-token ratio 
which is computed by dividing the num- 
ber of different words (types) by the total 
number of words (tokens) in a given 
sample. In this study the ratio was com- 
puted for each one-hundred-word seg- 
ment and the twenty-eight segmental 


% Table 1 in Appendix C of the manuscript 
copy of this report on file at the State Univer- 
sity of Iowa Library contains an alphabetical 
word list showing the number of freshmen and/or 
schizophrenics who used each word, and the fre- 
quency of its occurrence in each group. Words 
starred in the list are words which were con- 
sidered neologisms in the generally used sense 
of that term; that is, they were privately coined 
by the individuals who used them and are not 
used by other persons. The starred words of 
the freshmen are mainly slang terms essentially, 
although relatively unusual. 
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TTR’s obtained from each sample were 
averaged to secure a mean segmental 
TTR for each individual. An overall 
TTR was also obtained for each individ- 
ual by considering the 2800-word sample 
as a unit and dividing the number of 
different types in the entire sample by 
2800. (2) Grammatical Analysis; and (3) 
Type Frequencies. Statistical treatment 
of the data resulted in the following find- 
ings. 


Type-Token Ratios 


1. When the twenty-eight segmental 
T'TR’s for each subject were split at ran- 
dom into two sets of TTR’s, the mean 
for each random half computed, and the 
t-test for related measures applied, it was 
found that the difference between the 
mean segmental TTR’s yielded by the 
two random sets. was not statistically sig- 
nificant for the patients nor for the 
freshmen. 

2. The standard deviation of the 
twenty-eight segmental TTR’s for each 
subject was computed. When the F-test 
of the significance of the difference in 
variability was applied it was found that 
the schizophrenic patients showed sig- 
nificantly more variability in the number 
of types used per one-hundred-word seg- 
ment than did the freshmen. 

3. When the mean segmental TTR 
and the overall TTR for each subject 
were compared it was found that the 
overall I'TR’s were consistently lower 
for all subjects than the mean segmental 
TTR’s, bearing out the assumption that 
as an individual’s verbal output increases 
the rate of increase in the number of dif- 
ferent words he uses tends to decrease. 
There was some overlapping between the 
schizophrenic patients and freshmen on 
both the mean segmental TTR’s and 
the overall TTR’s, the range of values 
for mean segmental TTR’s being .4600 
to .7450, and .6708 to .7357, and for 


overall TTR’s .1850 to .3932 and .2689 
to .4079 for the patients and freshmen, 
respectively. 

4. Group mean segmental T'TR’s were 
obtained by averaging the mean segmen- 
tal TTR’s for the individuals within 
each group. The mean segmental TTR 
for the schizophrenic group was found to 
be significantly lower than the mean 
segmental TTR for the freshmen. The 
variance of the distribution of mean 
segmental TTR’s for the patients was 
found to be significantly greater than 
the variance of the corresponding dis- 
tribution for the freshmen. When the 
analysis of the significance of the differ- 
ence between the group mean segmental 
TTR’s was extended by using ¢ to estab- 
lish limiting values of the true mean for 
each group, it was found that there was 
no overlap between these ‘confidence in- 
tervals’ for the two groups at the one per 
cent level of confidence. 

5. Comparisons were made to deter- 
mine the effect of certain variables, 
among the schizophrenics, on their mean 
segmental TTR’s. These intra-group 
comparisons indicated that differences in 
intelligence test scores, level of educa- 
tional attainment, and duration of con- 
finement in the hospital had relatively 
insignificant influence on the TTR’s for 
the patients, and did not adequately ac- 
count for the differences between the 
schizophrenic patients as a group and 
freshman students as a group. 

6. Written language samples obtained 
in this study were compared with spoken 
language samples obtained by Fairbanks 
(12) from schizophrenic patients and 
freshman students. The mean TTR’s for 
both types of subjects run considerably 
higher for written than for spoken lan- 
guage. This finding may be attributed 
to the fact that, generally speaking, an 
individual’s written language is a more 
finished product, permitting more alter- 
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ing and rearranging of the words used, 
than is his spoken language. 

7. In regard to overall TTR’s it was 
found that the mean overall TTR for 
the schizophrenic patients was signifi- 
cantly lower than the mean overall TTR 
for the freshmen, and that the variability 
in overall I'TR’s for the patients was 
significantly greater than the variability 
in overall TTR’s for the freshmen. 
When ¢ was used to set limiting values 
of the true mean overall TTR for each 
group, there was slight overlap in the 
intervals for the patients and freshmen 
at the one per cent level of confidence, 
but there was no overlap in these inter- 
vals at the two per cent level of confi- 
dence. 

8. Differences between the sexes for 
the two groups in regard to mean seg- 
mental I'TR’s and overall TTR’s were 
not statistically significant, nor was the 
variability for either sex significantly 
greater than the variability for the other 
sex with regard to either of the meas- 
ures. 

g. Correlation between mean segmen- 
tal and overall TTR’s resulted in a 
Pearson product-moment correlation co- 
efficient of .62 for the patients and .62 
for the freshmen. For all subjects the r 
was .71. 


Grammatical Analysis 


1. Differences between schizophrenics 
and freshmen in relative frequency of 
usage of each of five grammatical classi- 
fications (adjectives, adverbs, nouns, pro- 
nouns, and verbs), expressed as percent- 
ages of the total number of words used, 
were not statistically significant, with the 
possible exception of the difference be- 
tween the groups in relative frequency 
of usage of nouns, which was significant 
at the five per cent level of confidence, 
the patients using more nouns than the 
freshmen. 


2. Differences between males and fe- 
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males within each group in relative fre- 
quency of usage of the grammatical cate- 
gories tested were not statistically sig- 
nificant for schizophrenic patients nor 
for freshmen. 

3. Comparison with Fairbanks’ data 
shows that there marked increase 
in percentage of nouns, adjectives, prepo- 
sitions, and articles, for both groups, 


is a 


and in conjunctions for schizophrenics, 
in written over spoken language, and an 
increase in 
verbs, 


percentage of pronouns, 


adverbs, and interjections, for 
both groups, and in conjunctions for 
the freshmen, in spoken over written 
language. 

4. Ratios of adjectives to verbs, ad- 
jectives to nouns, and adverbs to verbs 
were generally higher for the freshmen 
than for the patients, the difference with 
regard to the adjective-verb quotient be- 
ing the greatest; this difference fell very 
slightly short of significance at the five 
per cent level. 


Type Frequencies 

1. Eighty-three words were common 
fre- 
quently used words for both schizophren- 
ics and freshmen. 


to the lists of one hundred most 


2. The number of neologisms, i.e. pri- 
vately coined words, was fifty-seven for 
the schizophrenics, and five for the fresh- 
men. 

3g. When the vocabularies for each 
group were considered from the point of 
view of the number of types used to 
make up a certain per cent of the total 
number of words, it was found that ten 
types made up slightly more than twenty- 
five per cent of the tokens for each 
group, and that ninety-five types made 
up fifty per cent of the tokens for the 
schizophrenics, ninety-six types 
made up fifty per cent of the tokens for 
the freshmen. 


while 


4. Differences in the frequencies with 
which certain types occurred in the 
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spoken language of schizophrenics and 
freshmen reported by Fairbanks were not 
found in the written language of these 
two groups. 


Conclusions 


Of the measures used in this study the 
type-token ratios appear to offer the most 
fruitful means of differentiating quanti- 
tatively written language samples of the 
type investigated. With the exception of 
the adjective-verb quotient, and perhaps 
certain other ratios of parts of speech, 
the grammatical analysis did not prove 
useful in this respect. From the results 
reported by Fairbanks (12) as to the fre- 
quency of certain types in spoken lan- 
guage, and from observations of clinical 
manifestations of ego-centricity, negativ- 
ism, and frequency of neologisms, the 
prediction might logically have been 
made that an investigation of type fre- 
quencies would provide a quantitative 
differentiation of the language of the 
groups studied. However, the results of 
the analysis were contrary to this pre- 
diction. It is possible that the formality 
of the writing situation offers a possible 
explanation of the relative infrequency 
of self-reference terms, for example, in 
the written language of schizophrenics. 
However, the fact still remains that the 
freshmen students used relatively more 
first person singular pronouns, while the 
patients used relatively fewer such pro- 
nouns, in written as compared to spoken 
language. Two other considerations may 
be mentioned in this respect. It could be 
postulated that the task assigned the sub- 
jects in this study, that of writing a “life 
story”, would tend to increase the fre- 
quency of reference to self. This may 
actually have operated to increase the 
frequency of self-reference for the fresh- 
men, but for the schizophrenic patients 
this effect may have been counteracted 
to a large extent by their tendency to 
enumerate, and to get ‘off the track’ in 


fe 


recounting their life histories by describ- 
ing certain places, events, or things, with 
little or no reference to their own rela- 
tion to such places, events, or things. 
This was particularly noticeable in the 
writing of some of the patients, one of 
whom went to great pains to describe 
how one (or you) may “bake bread”, 
“can apples’, “teach a class in geogra- 
phy”, etc., but with almost no reference 
to self involved in such descriptions. It 
appears obvious from the lack of differ- 
entiation between the two groups in 
terms of the frequency cf specific types, 
that further investigations into this prob- 
lem will require the formulation of cer- 
tain other measures designed to offer a 
means of evaluating the ‘adequacy’ of 
the language from a different standpoint. 

Insofar as this is a study of ‘psycho- 
pathological’ language on the one hand, 
and ‘normal’ language on the other, 
certain conclusions may be drawn as to 
the differences between these types ol 
language. The ‘normal’ subjects investi- 
gated in this study appear to have a 
more highly differentiating language 
structure in that they use more adjectives 
per noun, more adverbs per verb, and 
more adjectives per verb, than do the 
schizophrenics, This may be interpreted 
to mean that on the whole they define, 
modify, and restrict their language in 
such a way as to make it more accurately 
representative of the actualities which 
they are attempting to symbolize. The 
assumption that ‘normal’ language struc- 
ture is more highly differentiated is fur- 
ther substantiated by the fact that the 
‘normal’ subjects have higher type-token 
ratios indicating that they use more dif- 
ferent words in producing a given verbal 
output, than do the schizophrenic pa- 
tients. 

The language of schizophrenics does 
not appear to be differentiated from 
‘normal’ language in terms of the specific 
most frequently used words. The vocabu- 
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laries of the two groups in this study 
appear to be very similar in that there 
is an overlap of eighty-three words be- 
tween the lists of one hundred most fre- 
quently used words for each group. The 
only differentiating feature which a study 
of the vocabulary pointed to was the rela- 
tive frequency of neologisms in the lan- 
guage of schizophrenics, as compared to 
the frequency of their occurrence in the 
language of freshmen. 

As a preliminary investigation this 
study has provided a quantitative dif- 
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ferentiation of language of different 
types of individuals, and points the way 
to further research with particular refer- 
ence to determining the degree of cor- 
relation between these measures and 
other pertinent variables, and to a com- 
prehensive study of language develop- 
ment. Further development and modifi- 
cation of such quantitative measures may 
provide a means of constructing scaled 
continua with reference to which any 
given language sample might be evalu- 
ated. 
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I. INTRODUCTION 


HE PRESENT INVESTIGATION is con- 
‘Lae with the relation of certain 
language variables to (1) the length of 
sample from which they are derived and 
(2) certain psychologically pertinent fac- 
tors. In general, the language measures 
employed are based on a count of the 
number of different words (types) and 
the relationship of such measures to the 
total number of words, and to the fac- 
tors of I.Q., C.A., locality (city, town, 
rural), and sex. Similar measures based 
on parts of speech categories and their 
relationship to I.Q., C.A., locality and 
sex will be reported. Finally the relation- 
ship of the reliability of these measures 
to the length of samples from which they 
are derived will be given attention. 

Certain previous investigations have 
been concerned with closely related prob- 
lems. To begin with, Carroll (3) has pre- 
sented an equation describing the rela- 
tion of the number of different words 
(D) to the total number of words (N) in 
a sample of language. A necessary condi- 
tion to Carroll’s formulation of this rela- 
tionship is that a specified relationship 
hold between the frequency of a given 
word in a language sample and its rank 
in order of decreasing frequencies. Zipf 
(15) discovered that when he plotted 
frequency of a word against the number 


* This study was done in the Department of 
Psychology at the State University of Iowa as a 
dissertation in partial fulfillment of the require- 
ments for the degree of Doctor of Philosophy. It 
is part of a program of research on language 
behavior. The study was directed by Drs. Wen- 
dell Johnson and Don Lewis. Funds and assist- 
ance were provided by the Federal Work Projects 
Administration in connection with Iowa WPA 
Projects 4892 and 5960. 
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of words having that frequency on loga- 
rithmic co-ordinates, the points approxi- 
mated a straight line except for the few 
most frequently occurring words. From 
this fact he formulated the harmonic 
series law of word distribution, in which 
he states that the most frequent word in 
a large sample of language makes up 
14, of the sample, the second most fre- 
quent word 14, of the sample, the third 
most frequent word 14, of the sample, 
etc. This formulation can be put in the 
form of the equation 


F = —— 
10R 
in which F is the frequency of occurrence 
of any given word in a language sample, 
R is its rank in order of decreasing fre- 
quencies, and N is the total number of 
words in the sample. 

Skinner (13) has also presented results 
pertaining to the relationship between 
F and R. In analyzing the results ob- 
tained from 1,000 responses to his verbal 
summator, he plotted ranks of words in 
order of decreasing frequencies (R) 
against frequency (F), expressed as a 
percentage of the total sample, on log- 
arithmic coordinates, and found the 
points tended to fall on a straight line. 

A deviation from linearity was again 
noted in the more frequently used words. 
Skinner (13) also reanalyzed the Kent- 
Rosanoff (8) data on free association re- 
sponse words in the same manner. He 
found that when the rank order of words 
in terms of mean frequency per thousand 
was plotted against mean frequency per 
thousand on logarithmic coordinates, the 
resulting curve was approximately linear 











eens eis: tienen ati OE 


























78 JOHN W. CHOTLOS 


for the 100 responses most likely to occur. 
The equation 


300 
f— 





R1.29 

where f is the frequency with which a 
given association will occur in 1,000 re- 
sponses and R is its rank in terms of 
mean-frequency per thousand, he finds 
to be descriptive of the 75 words having 
the strongest first associations in the 
Kent-Rosanoff list. He states that this 
formula is slightly less accurate for the 
total sample, and his calculated and ob- 
served points appear to agree satisfac- 
torily. However, he states that this equa- 
tion has little practical significance since 
the frequency and rank of a word must 
be ascertained before the equation can 
be used. Nevertheless, he feels that it has 
an important bearing on theories of lan- 
guage. 

Carroll argued that if one accepts the 
harmonic series law of word frequency 
distribution, i.e., 

N 

F =—— 

KR 
where F is the frequency of any word in 
a language sample, R its rank in order 
of decreasing frequencies, N the total 
number of words in the verbal output 
sample, and K is a constant which is an 
indirect index of diversity, then it can 
be demonstrated that the following equa- 
tion holds: 


D= va (0.423 + K — log, N + log, K) 
K 
where D is the number of different words 
in a sample, N the total number of words 
in that sample and K is an empirically 
determined constant. This equation, if 
it can be shown to be applicable in gen- 
eral, has very important implications 
with regard to language since, in the 





first place, if D is known for a specified 
N, predictions can be. made to other N’s, 
and, secondly, the nature of the curve 
allows a determination of a maximum 
value of D, a value which can be cor- 
related to a given type of vocabulary of 
the individual. Carroll tested this equa- 
tion with a verbal output sample ob- 
tained by means of the verbal summator 
technique, and on several language sam- 
ples from literature, and found the em- 
pirical points to fall very near to the 
computed curve. 

The language samples which formed 
the protocols of these investigations into 
the relationship between D and N and 
between F and R, have been accumu- 
lated from different sources and massed 
into one language sample or are the 
product of verbally proficient writers. 
Thorndike (14) has emphasized that if 
language is to be viewed as behavior, the 
motivation, backgrounds, the individual 
characteristics of the writers or speakers 
must be taken into consideration. He 
further suggested that the relationship 
between F and R reported by Zipf may in 
some measure be a statistical artifact 
produced by combining language from 
varied sources, such combination result- 
ing in a loss of individual variation. In 
the light of this criticism it seems desir- 
able to apply the mathematical formula- 
tions of these relationships to samples of 
language which are the product of but 
a single individual, in order to test their 
adequacy more fully. 

A second point of interest in the con- 
sideration of previous studies is that 
various attempts have been made to re- 
late the number of different words in a 
sample to psychologically pertinent fac- 
tors. Fairbanks (5) working with spoken 
language and Mann (9) with written lan- 
guage compared superior university 
freshmen and schizophrenic patients in 
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terms of the mean percentage of differ- 
ent words per 100-word segment, i.e., the 
100-word type-token ratio. Both inves- 
tigators found the mean type-token ratio 
for superior freshmen to be significantly 
greater than for schizophrenic patients, 
indicating a wider vocabulary range for 
freshmen than for schizophrenic pa- 
tients. Fairbanks suggests that for spoken 
language, there might be a positive cor- 
relation between the 100-word type- 
token ratio and intellectual level. On the 
other hand, Mann (g) found that differ- 
ences in intelligence test scores, level of 
educational attainment, and duration of 
hospital confinement had relatively little 
influence on the type-token ratio of pa- 
tients in terms of accounting for differ- 
ences between her two groups. In neither 
study were significant sex differences 
found. Fossum (6), studying spoken lan- 
guage obtained from junior college stu- 
dents in a regular speech class, found by 
means of a correlation technique that 
the 100-word type-token ratio based on 
18 segments appeared to be related to 
parental occupation, correlation of .56, 
and to speaking rate, correlation of 
—.45, but not to vocabulary as measured 
by the Nelson-Denny Reading Test, cor- 
relation of .og, nor to intelligence as 
measured by the percentile score on the 
Ohio State Psychological Test, correla- 
tion of .og. Fossum found no sex differ- 
ences in type-token ratio measures in his 
group. 

Thirdly, counts of the number of 
words in parts of speech categories have 
been made by Fairbanks (5) and Mann 
(9). They related these counts to other 
variables under consideration. Fairbanks 
(5) found that for spoken language there 
were differences in the use of various 
parts of speech between her freshmen 
and schizophrenic groups. The schizo- 
phrenics used proportionately more pro- 


nouns and verbs and proportionately 
fewer nouns and articles. On the other 
hand, for written language, Mann (9) 
found that the results of the grammatical 
parts of speech count were not signifi- 
cant, although there seemed to be a 
tendency for the patients to use more 
nouns than the freshmen, a result which 
does not agree with the comparable re- 
sult obtained by Fairbanks. 

Fourthly, Fossum (6) has attempted to 
relate the reliability of the type-token 
ratio to the length of spoken language 
sample. He found the correlation be- 
tween 100-word type-token ratios for the 
two halves of 1,800-word samples to be 
.58 and he estimated by means of the 
Spearman-Brown prophecy formula that 
a sample of 14,000 words would be 
needed to give a reliability coefficient of 
.Q5- 

Many important facts about language 
have been reported, and it is of un- 
doubted importance to know whether 
the relationships already reported will 
hold for individual language samples. 
Many investigators have used verbal out- 
put samples in which the individual 
characteristics of the writers and speakers 
have been lost through massing of the 
data. In other instances, where individ- 
ual variations have been of major inter- 
est in the investigation, the samples of 
individuals have been highly selected. It 
is proposed in this study to investigate 
language characteristics of individual 
verbal output samples in which the pop- 
ulation sampled will allow for consid- 
erable generalization of the results. ‘The 
question of whether or not individual 
verbal output samples will bear out the 
equations descriptive of massed language 
data is an important one. Furthermore, 
the relationship between language meas- 
ures and I.Q., C.A., sex, etc., appears to 
need further investigation in popula- 
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tions less highly selected in these charac- 
teristics than the ones which have been 
so far reported. 


Il. THE PROBLEM 


The object of this study may be ori- 
ented around the analysis of the number 
of different words (D) as a function of 
the total number of words (N). This re- 
lationship may be symbolized by the 
formula. 


D = f(N) 


In this equation it is apparent that we 
can hold N constant and study D in re- 
lation to other variables, we can study 
the variation in D with concomittant 
variation in N, and finally we can study 
the variation in D and N in relation to 
other variables, such as intelligence test 
score, age, etc. The basic unit of analysis 
is the language sample of a single indi- 
vidual. With this introduction the pur- 
pose of this study can be summarized in 
the following statements and questions: 


1. To test empirically the equation derived 
by Carroll, namely, 
N r 
D = — (.425 + K — log, N + log, K) 
K 


where D is the number of different 
words in a sample of length N and K 
is an empirical constant to be deter- 
mined from the data. It is a further 
purpose to test the assumptions under 
which this equation was developed. 

2. Can the relationship between D and N 
be expressed by some empirically deter- 
mined curve? If such a curve can be de- 
termined can the constants in this curve 
be given any rational meaning? 

3. Does D for specified N’s differentiate 
I.Q. groups, age groups, location 
groups, and sex groups? And what is 
the extent and direction of these dif- 
ferences? 

4. Do sections of the language samples, 
categorized by parts of speech, reveal 
any relationships or differences which 
are not apparent in the sample as a 


whole? How are the parts of speech 
which go to make up the total sample 
interrelated? 

5. What is the minimum size of sample 
that can be drawn to reveal the rela- 
tionships and differences under inves- 
tigation? 

III. SUBJECTS AND PROCEDURES 

As part of a remedial education sur- 
vey, sponsored by the Iowa Child Wel- 
fare Research Station and financed by 
the Federal Work Projects Administra- 
tion, approximately 1,000 public school 
children wrote manuscripts of 3,000 
words each under conditions to be speci- 
fied below. The collection and prelimi- 
nary analysis of these manuscripts was 
carried out by Work Projects Adminis- 
tration personnel under the supervision 
of persons with background training in 
psychology, who had been given special 
training for this particular assignment.? 
The survey operated for a period of 
about two years in five counties of the 
state of Iowa. These counties are dis- 
tributed throughout the state in such a 
way that no two counties were adjacent. 
Each county survey was operated as a 
unit, coordination being achieved by a 
State-wide supervisor stationed at the 
university. In each county all schools, 
including the one-room rural schools, 
were invited to participate in the pro- 
gram. 

Unit supervisors were instructed to 
collect 3,000-word language samples for 
an allotted number of pupils in their 
respective units. As it took several hours 
for a child to write the number of words 
required, extensive cooperation from the 
school administrators and teachers was 


? Professor George D. Stoddard served as gen- 
eral director of the survey, of which the lan- 
guage study was a part; Dr. Wendell Johnson 
was the technical director; Dr. C. Ecco Aber- 
mann and Mr. George Wischner served succes- 
sively as statewide supervisors; and the present 
writer was the project statistician, 
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TABLE I 


Factorial design of experimental sample 





Male 














Female 
City Town Rural City Town Rural 
_ eK 
EY hg Age 1* 2 2 2 2 2 2 
I Age 2 2 2 2 2 
Age 3 2 2 2 2 2 2 
1.Q. Age 1 2 2 2 2 2 2 
2 Age 2 2 2 2 2 2 
Age 3 2 2 2 2 
1.Q. Age 1 2 2 2 2 2 
3 Age 2 2 2 2 
Age 3 2 2 2 2 2 2 





* Age 1—149 months and under. 
Age 2—150 to 179 months. 
Age 3—180 months and over. 


Or 


1.Q. 1— 89 and under. 
1.Q. 2— go to 109. 
1.Q. 3—110 and over. 


*** The numbers refer to the number of randomly selected subjects in the cell. 


necessary. The plan called for collecting 
an equal number of samples from city, 
town and rural school children, from 
equal numbers of boys and girls, and an 
equal number from each grade from 
four through twelve. Localities with a 
population of 25,000 or over were called 
cities, other localities and consolidated 
schools were considered as town schools, 
and rural schools of the one-room variety 
were considered as rural for purposes of 
this study. Since, as a rule, the one-room 
rural schools have only eight grades, it 
was necessary to classify town school pu- 
pils who had a rural school background 
and whose parents were farmers, as rural 
in order to fill out the rural categories 
at the older ages. In collecting this sam- 
ple the pupils were matched by sex for 
grade, age (within six months), 1.Q. 
(within five I.Q. points) and socio-eco- 
nomic level (within the limits of 1920 
U. S. census occupational classification 
system). No pupil under eight years nor 
over eighteen years of age was included 
in the sample. 

The writing was done under the su- 
pervision of a worker who remained in 


the classroom throughout the writing ses- 
sion. Writing sessions averaged about 
forty minutes in length and, on the 
whole, four or five writing sessions were 
required for a child to complete his as- 
signed task. 

The worker in charge read the follow- 
ing instructions before the children be- 
gan to write: 


“You are to write about anything you want 
to write about. Just make it up as you go 
along. That is, don’t write anything you have 
memorized such as stories or poems. Just 
start with the first thing you think of and 
try to keep on writing steadily.” 

If a child stopped writing for longer 
than five minutes or complained that he 
couldn't go on, the worker was in- 
structed not to tell him what to write 
about, but to tell him to write on what- 
ever he was thinking about. No positive 
suggestion as to topics was allowed. 
Legibility of the manuscript was empha- 
sized and speed was not encouraged. 
Each day’s writing was handed in to the 
monitor at the close of the session. ‘The 
worker counted the number of words 
written, entered the count in his record 
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TABLE 2 
Means and standard deviations of: distributions 
of ages in months for the total group and for 
the main sub-groups 


TABLE 3 
Means and standard deviations of distributions of 
Otis intelligence test scores (I.Q. units) for the 
total group and the main sub-groups 














Standard ; Standard 
Group Mean Phevintion Group Mean Deviation 
Age Groups Age Groups 
149 months and under 129.806 12.684 149 months and under _ 101.278 14.251 
150 to 179 months 164.389 8.473 150 to 179 months 101.528 14.297 
180 months and over 189.639 7.779 180 months and over 99.028 14.937 
1.0. Groups I.Q. Groups 
89 and under 163.917 26.186 89 and under 83.833 3.670 
go-109 162.083 26.319 go-I109 101.861 4.461 
110 and over 157.833 26.455 110 and over 116.139 4.596 
Location Groups Location Groups 
City 165.194 24.120 City 101.056 14.012 
Town 159.861 28.136 Town I00.1II 14.240 
Rural 158.778 26.469 Rural 100.667 13.383 
Sex Groups Sex Groups 
Male 160.796 25.887 Male 100.556 14.453 
Female 161.759 27.020 Female 100.667 13.300 
Total Group 161.278 26.443 Total Group 100.611 13.888 





and dismissed the subject when he had 
reached the prescribed quota. 

From this basic sample of approxi- 
mately 1,000 language samples, an ex- 
perimental sample of 108 was selected 
to conform to the factorial design in 
Table 1. Since a complete record was 
available on each child it was possible 
to sort the larger sample of manuscripts 
into the fifty-four cells of the design and 
select at random two subjects for each 
cell. There was no matching by sex in 
the experimental sample as was the case 
in the survey sample. Intelligence was 
tested by means of the Otis Quick-Scor- 
ing Mental Ability Tests. The Alpha 
test was administered to pupils in grades 
one through four, the Beta test to pupils 
in grades five through nine, and the 
Gamma test to pupils in grades ten 
through twelve. In classifying the sub- 
jects according to the design in Table 1, 
no distinction was made between the 
various forms of these three tests. Ages 
were computed as of the day the chil- 
dren started writing. Criteria for the 





locality levels of the design are the same 
as those mentioned above for the collec- 
tion of the basic sample. The design per- 
mits a distribution of 36 subjects at each 
of three I.Q. levels: (1) 89 and under, 
(2) go to 109, and (3) 110 and over; 36 
subjects at each of three age levels: (1) 
12 years, 5 months and under, (2) 12 
years, 6 months to 14 years, 11 months, 
and (3) 15 years and over; 36 subjects at 
each of three locality levels: (1) city, (2) 
town, and (3) rural; and 54 subjects in 
each of the sex groups. Furthermore, 
many combinations of I.Q., age, locality 
and sex levels are possible. 

Means and standard deviations for dis- 
tributions of I.Q. and age for the total 
experimental sample and for the main 
sub-groups, i.e., in terms of I.Q., age, 
locality and sex, are presented in Tables 
2 and 3g. 

Following the collection of the lan- 
guage samples, the manuscripts were 
typed and edited. The definition of a 
word, it should be realized, is crucial in 
a study of this type. Quite a bit of free- 
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x 


dom is permitted in defining a unit of 
language and the way in which the unit 
is defined is necessarily a condition that 
is important in connection with any 
statements made about language phe- 
nomena. For this reason the rules for 
editing the manuscripts are presented in 
full. These rules define the fundamental 
language unit better than any formal 
definition could. The following rules 
were followed in editing the samples: 


1. Type all words exactly as they are writ- 
ten by the subject. Record each cor- 
rection by writing it in parenthesis 
after the word for which it is a cor- 
rection. 

2. Correct each misspelling, recording the 
word as spelled by the subject and 
writing the correction after it, in ac- 
cordance with (1) above. 

Classify as a misspelling any word which 

as spelled by the subject does not con- 

stitute a standard English word (cur- 
rent edition of the Century Dictionary 
to be used as authority) or a recog- 
nizable ‘slang’, nonstandard word (rec- 
ognizable to the present investigators). 

3. Classify as a substitution and correct 
in accordance with (1) above, any of 
the following. 

a. Any correctly spelled homonym or 
an apparently ‘intended’ word; e.g., 
“their” substituted for an appar- 
ently intended “there’’, “bare” for 
“bear”, “four” for ‘for’, etc. Judg- 
ment in such cases will involve rea- 
sonable interpretation of context. 

b. Any correctly spelled non-homony- 
mous substitution which apparently 
distorts the ‘intended’ sense; e.g., “‘of 
you own” for “of your own’, “is 
would be” for “it would be”. Judg- 
ment in such cases will, again, in- 
volve reasonable interpretation of 
context. 

4. Do not insert any word apparently or 
obviously omitted by the subject. For 
example, if the subject writes, “It 
would fun to play ball,” do not in- 


sert the word “be” at the point where . 


the subject obviously omitted it. 
5. Record slang or non-standard words as 


written by the subject. When a slang 
or non-standard word has a standard 
equivalent, record this equivalent in 
parenthesis after the slang word; e.g., 
write “sneaked” in parenthesis as a cor- 
rection for “snuk”. Any slang or non- 
standard word having no standard 
equivalent is to stand as written by 
the subject and misspellings of such 
words when recognizable are to be re- 
corded in accordance with (1) above. 


9. Any proper name which consists of 


more than one word is to be counted 
as one word; e.g., “John Jones” is one 
word; “East St. Louis” is one word; 
but “East St. Louis, Illinois’ is two 
words, since they constitute two proper 
names, the name of a city and the 
name of a state; “The Chicago and 
Northwestern Railroad” are three 
words since (1) “the” is never to be 
regarded as an integral part of a 
proper name, always being counted as 
a separate word and (2) any class-name 
to which a proper name is attached 
is to be counted as a separate word; 
e.g., “railroad”, “hotel”, “theatre”, 
“street”, etc., even in such an exam- 
ple as “the Hotel Roosevelt’, “the” 
and “Hotel” are to be counted as sep- 
arate words. A proper name is one that 
designates the sole bearer of the name, 
as: there is only one “Chicago and 
Northwestern” railroad, only one 
“Great Altantic and Pacific’ tea com- 
pany, only one “General Motors’ cor- 
poration, etc. The names given above 
in quotes, therefore, are proper names 
and each is counted as one word. In “A 
1940 Multi-Motored Amphibian P45 
Boeing Transport”, on the other hand, 
the various words are qualifying adjec- 
tives; there are many Transports—Boe- 
ing names one type, and it, rather than 
“Boeing Transport” is a proper name 
in this case; there are many Boeing 
Transports, and P45 merely serves as 
an adjective—there might be P44, P46, 
etc. Again “Amphibian” serves as an 
adjective, and so for the terms, “a”, 
“1940” and “multi-motored”. In the 
example of “Dubuque Senior High 
School”, “Dubuque” is an adjective, of 
course; ‘Senior High School”, however, 
is not one word in the same sense 
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that “Chicago and Northwestern” is 
one word; “Chicago and Northwest- 
ern designates the only railroad that 
goes by that name, but there are 
thousands of senior high schools; there- 
fore, “senior” and “high” are to be re- 
garded as adjectives; ‘““‘Dubuque Senior 
High School” is to be regarded as four 
words. 

“Mrs.”’, “Mr.”’, “Miss” and other modes 
of address are to be counted as separate 
words and not as integral parts of 
proper names; e.g., “Mr. John Jones” 
are two words. 

Titles are not integral parts of proper 
names, but are to be counted as sep- 
arate words; e.g., “Doctor Jones” are 
two words; “Senator Hill” or “Profes- 
sor Smith” are two words. 

Abbreviated titles which consist of 
more than one unit, e.g., M.D., or 
Ph.D., or unabbreviated titles which 
consist or more than one word, e.g., 
“Speaker of the House” or “Dean 
Emeritus”, are to be counted as sin- 
gle words. 

Any number is to be counted as one 
word and all figures are to be written 
or changed to longhand words. “One”, 
“twenty-seven”, “one thousand  six- 
teen’ are each a single word. Where 
time is denoted in numbers, it should 
be counted as a number; e.g., “7:35,” 
write as “seven thirty-five” and count 
as one word. 

Where street numbers are denoted, 
write as customarily spoken: e.g., “1220 
Harrison Street’ write as “twelve 
twenty Harrison Street”. 

When numbers are placed at the be- 
ginning of sentences for no obvious 
reason they are not to be included in 
the typewritten copy and are not to 
be counted. For example, in one or 
two cases it was noted that the sen- 
tences had been numbered by the 
child. Such numbers are not to be 
counted. 

Contractions are recorded as written: 
e.g., “didn’t” is not to be changed to 
“did not”; “didn’t” is one word. 
Record abbreviations as written by the 
subject, with full term in parenthesis. 
Hyphenated words properly hyphen- 
ated (Century Dictionary to be used as 





authority) are to be counted as single 

words; e.g., “hitch-hiker’’ is one word. 

‘Two words improperly hyphenated are 

to be counted as two words. Correc- 

tions in such cases are to be made ac- 
cording to instructions given above. 

11. Any two words, as corrected and tabu- 
lated, are different unless spelled ex- 
actly alike, except: 

a. Plurals and possessives, and contrac- 
tions involving apostrophes, are to 
be differentiated even though they 
are spelled alike. 

b. Any word which begins with a cap- 
ital letter solely by virtue of its 
place in the sentence is not to 
be classified as different from a word 
spelled as it is in all other respects. 

12. All recognizable words are to be 
counted and tabulated except in the 
case where some symbol is used to in- 
dicate a previously written word. These 
symbols are not to be counted as that 
word. For example, 

“John is going to town tonight. 
gs dvi “school. 

be late.” 

There are nine written words and only 

nine are to be counted. 

13. Sentences are to be left as the child 
has written them. Do not change the 
pronoun to agree with the noun in 
the following example, “Ruth has one 
side on his paper’, nor change the 
tense in this example, “It is raining last 
night’. Count these as they stand. 


“e “e ‘é sé 


After the manuscripts were typed and 
edited, the types and tokens were re- 
corded and tabulated separately for each 
manuscript. This tabulation made it pos- 
sible to abstract the following language 
measures: 


1. The number of types in any 1oo-word 
segment, 500-word segment, 1000-word 
segment, and in the total 3,000 words. 
Thus, 30 measures from 100-word seg- 
ments, six measures from 500-word seg- 
ments, three measures from 1,000-word 
segments and one measure for the total 
manuscript were computed. For each 
subject these were averaged to give the 

mean number of types in 100, 500, and 
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1,000-word segments, respectively. The 
mean number of types in a specified 
segment can be symbolized by D, with 
a subscript to denote the size of the 
base from which it is computed. The 
four measures described above can be 
symbolized by D D D. ... and 


100’ 500’ 1,000 


3,000° 
The type-token ratio for one-hundred- 


word, five-hundred-word, one-thousand- 
word and_ three-thousand-word  seg- 
ments. The type-token ratio can be de- 
fined as the mean percentage of types 
in any specified segment. If the number 
of tokens is symbolized by N, then the 
type-token ratio can be symbolized by 
D; 

—, where the subscript 7 specifies the 
Ni 

size of the sample on which the type- 
token ratio was based. In our case, the 
following type-token ratios were com- 
puted: 














D0 Ds D, 000 
Rw = R50 = R, 000 — 
100 500 1,000 
Ds, 000 
R; 000 
3,000 


in which R is a symbol for type-token 
ratio. 

R and D, as defined above are equiv- 
alent measures so long as the number 
of tokens, N, on which they are based 
is equal for the different individuals 
in a distribution. However, in in- 
stances where a distribution is made 
up of measures based on a varying N, 
the two measures are not equivalent. 
In this study, where R and N gave 
equivalent measures, R was preferred 
because it made comparison with re- 
sults of other studies possible. 

The cumulative type frequency curve 
is obtained by cumulating the types 
added in each successive 100-word seg- 
ment. It is the curve that results when 
D, the number of types, is computed 
as a function of N, the number of 
tokens. Since each language sample was 
sectioned into thirty segments, there 
are thirty points available in the com- 
putation of this curve. 

The frequency and rank of each type 
was computed from the data. The fre- 
quency of a type is the number of 


~I 


times it occurred in the language sam- 
ple. A ranking of the types, in whole 
number steps, and in order of decreas- 
ing frequency, was made. Types of 
equal frequency were given an aver- 
age rank number. The numerical po- 
sition of a word in this sequence is 
its rank. The frequency of any type 
is symbolized by F; and its rank by 
Ri. 

Following the computation of the 
above language measures, the language 
sample was split into four subsamples. 
The division was made on the basis 
of these parts of speech: nouns, verbs, 
adjectives, and adverbs. It did not seem 
profitable to analyze the pronouns, 
prepositions, conjunctions and articles 
at this time since these parts of speech 
are much more limited in the number 
of available types. An attempt was 
made to classify the words on a func- 
tional rather than on a formal basis. 
In each instance the function of the 
word in the context in which it was 
found determined its classification. For 
example, the word run in “He will 
run” and “She had a run in her stock- 
ing” are considered as two different 
types in this classification, the first a 
verb and the second a noun, whereas 
in the previously described definition 
they would be counted as one type ac- 
cording to editing rule number 11. 
Curme’s (2) text, A Grammar of the 
English Language, was used as a ref- 
erence and final authority in case of 
doubt. On the basis of this grammati- 
cal analysis these additional measures 
were abstracted. 


. The number of nounal, verbal, ad- 


jectival and adverbal tokens. 

The number of nounal, verbal, adjec- 
tival and adverbal types. 

The type-token ratio for nouns, verbs, 
adjectives and adverbs. This measure 
is not equivalent to (2) above since 
the number of tokens on which the 
number of types is based is not equal 
from individual to individual. Usually 
type-token ratios are not directly com- 
parable unless they are based on the 
same number of tokens for each indi- 
vidual, but in this instance, it is felt 
that distributions of type-token ratios 
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derived from a varying number of 
tokens can be justifiably used because 
the total number of tokens is the same 
in all the manuscripts. If the parts of 
speech type-token ratios are weighted 
by their respective number of tokens, 
their sum will be found to equal d, 
the number of types in the language 
sample. 

8. Percentage of nounal, verbal, adjectt- 
val and adverbal types. This measure 
was computed by summing all the 
nounal, verbal, adjectival and adverbal 
tokens for each individual and then 
finding the percentage which the sep- 
arate nounal, verbal, adjectival and ad- 
verbal types are of this total. 


IV. RESULTS 
RELIABILITY OF DATA 


It should be realized that the record- 
ing, tabulating, and counting operations 
in this study were unusually extensive. 
Attainment of absolute accuracy in a 
study of this type is both expensive and 
extremely difficult in a reasonable length 
of time. Errors were reduced to a mini- 
mum by having all operations done 
twice and by constant supervision of 
the workers. Further, the procedures for 
recording, tabulating, and counting the 
data were so set up as to make possible 
continuous checking throughout the op- 
erations. One set of verbal samples was 
tabulated independently by two units 
of the project in order to get some in- 
dication of the accuracy of the work. 
Forty verbal samples comprised this set 
and the correlation between the two 
counts of the number of types in each 
manuscript was .983. 

It will be recalled that our data con- 
sist of 3,000-word language samples, the 
individual words of which have been 
tabulated in such a fashion as to allow 
for the determination of the number of 
different words, types, in 100-word seg- 
ments, or in any segment which is a mul- 
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tiple of 100 words. In order to test the 
reliability of the type-token ratios the 
technique of correlating ‘split-halves’ was 
employed. The segments of each of the 
108 language samples were split into two 
halves, the first 1,500 words constituting 
one-half and the last 1,500 words consti- 
tuting the other. Mean type-token ratios 
for 100- and 500-word segments were 
computed for these two halves and the 
correlation between the halves com- 
puted. The product-moment correlation 
coefficient for the mean type-token ratios 
for 100-word segments was .829, and for 
the 500-word segment type-token ratios 
the correlation coefficient was .826, Since 
these type-token ratios were computed 
for only half of the 3,000 words, it is de- 
sirable to have some estimate of what the 
correlation would be if the whole sample 
were used as a basis for computing the 
ratios. Such an estimate can be made by 
means of the Spearman-Brown prophecy 
formula. Estimated reliability coefficients 
for the full length of 3,000 words are 
.g06 for the 100-word type-token ratio 
and .goq4 for the 500-word ratio. An as- 
sumption basic to the use of the Spear- 
man-Brown prophecy formula is that, in 
this case, the language sample be homo- 
geneous throughout its length in the 
above two measures. If the two halves 
are homogeneous, i.e., measure the same 
aspect of language, then one would ex- 
pect the mean type-token ratios for the 
group to be approximately the same for 
the two halves. These means were found 
to be 62.58 and 62.64 for the first and 
last halves, respectively, with regard to 
the 100-word type-token ratio, and 40.63 
and 40.65 for the first and last halves, re- 
spectively, with regard to the 500-word 
type-token ratio. On this basis, the as- 
sumption of homogeneity throughout 
the verbal sample for these two measures 
appears to be tenable. 
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It may be, however, that 3,000 words 
is an insufficient number to reveal any 
trends in the behavior of these two meas- 
ures. One long sample of 18,000 words 
was available and was used to test the 
hypothesis of homogeneity on a longer 
language sample. This sample was ob- 


population variance is obtained from the 
deviations of the individual type-token 
ratios and the other from the variation 
in the means. None of these three anal- 
yses proved significant, the F-ratios being 
less than one in the case of the 100-word 
TTR, 1.756 for the 500-word TTR and 


TABLE 4 
Type-token ratio measures of an 18,o00-word language sample 








Mean 100-word 


Mean 500-word 


Mean 1,000-word Mean 3,o00-word 





TTR* TTR rTR 
3,000-word 
Sub-sample 

I .27" 50.33 42.40 31.20 

2 69.13 48.37 40.80 29.57 

3 68.87 47.13 40.00 29.27 

4 70.13 48.60 41.07 30.37 

5 69.83 47.27 38.50 27.40 

6 70.40 45-97 36.40 24.10 





* TTR is an abbreviation for type-token ratio. 
** All type-token ratios are expressed as percentages. 


tained under the same conditions and 
rules as the shorter samples, except that 
the subject volunteered to do the task. 
Several subjects volunteered to write 
long samples, but only in this one in- 
stance was the task carried beyond the 
3,000 word quota. It is recognized that 
one subject, as such, has no statistical 
status and that no generalization what- 
ever can be made to the language be- 
havior of other children. It is offered as 
a particular case of language behavior 
and because it may be provocative of fu- 
ture leads. The child who wrote this long 
sample was fifteen years old, attended 
senior high school, had an I.Q. of 120, 
and came from a town school. The long 
sample was sectioned into six 3,000 word 
sub-samples and the 100, 500, 1,000, and 
3,000-word type-token ratios computed 
for each sub-sample. These results are 
presented in Table 4. Since the type- 
token ratios in the first three columns, 
i.e., for 100, 500 and 1,000 words, are 
means, it was possible to do an analysis 
of variance in which one estimate of the 


1.646 for the 1,000-word TTR. For sig- 
nificance at the five per cent level of con- 
fidence, F-ratios of 2.30, 2.54 and 3.20 
respectively, would be required. The 
3,000-word TTR could not be tested by 
this technique since there is but one 
measure of it in each sub-sample. 

On the basis of this analysis one would 
infer that the differences between the 
various sub-sample T'TR’s for 100, 500, 
and 1,000 words can be attributed to 
chance and that the hypothesis of 
homogeneity has not been discounted. 
However, this statement holds only if 
the six sub-samples are randomly se- 
lected from a population of such sub- 
samples. Since there appears to be a 
downward trend in the magnitude of 
these type-token ratios with an increase 
in the number of words written, except 
in the case of the 100-word type-token 
ratio, this assumption of randomness 
may not be fulfilled. If the sub-samples 
cannot be assumed to be randomly se- 
lected, the results of the F-test can be 
ignored and the data interpreted in 
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terms of a trend. It is the opinion of 
this investigator that in this one instance 
the results presented in Table 4 are 
indicative of a trend toward a reduction 
in the use of types with an increase in 
the number of words written. With the 
exception of the 100-word TTR, the 
T'TR’s may not be considered as homo- 
geneous throughout the length of this 
sample of language. Factors operating to 
produce such an effect may be (1) reduc- 
tion in the number of topics available 
to the child tending to a greater repeti- 
tion of types related to topics already 
discussed, (2) change in motivating con- 
ditions, such as loss of interest, boredom, 
competition with other activities, and 
(3) an adaptation to the writing situa- 
tion tending to produce stereotyped be- 
havior. Further, the fact that the 100- 
word TTR does not show this trend 
may be indicative of chance factors op- 
erating in the other three ratios, or it 
may indicate that the various type-token 
ratios are not measuring the same aspect 
of language. 

In any event, it may be considered as 
demonstrated that for 3,000 words the 
100-word and TTR’s are 
homogeneously distributed throughout 
the length of the sample, but that, on the 
basis of one case, this homogeneity may 
not be assumed to be necessarily present 
much beyond 3,000 words, except per- 
haps for the 100-word TTR’s. 

A problem closely related to that of 
reliability is posed by this question: 
“What is the minimum number of words 
that need to be sampled from one in- 


500-word 


dividual to obtain an adequate measure 
of his language behavior in terms of 
type-token ratios?” The answer to this 
question is of more practical interest 
than theoretical, since the number of 
words sampled has been, necessarily, ar- 
bitrarily determined. If a positive answer 
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can be made to this question much la. 
borious and time-consuming work en- 
tailed in language studies may be par- 
tially eliminated. 

Since no child wrote his full quota of 
3,000 words in one day and since ther 
appears to be practically no carry-over 
in any given child’s topics from day to 
day, it is felt that the first part of a 
child’s output could be considered as 
relatively independent of his last part. 
Correlations of T'TR’s for the first and 
last part of the sample, based on succes- 
sively larger numbers of words, should 
give an indication of the reliability ot 
the TTR as the base number of words 
is increased. For this purpose the follow- 
ing correlation coefficients were com- 
puted for 108 pairs of subjects: 


(1) TTR of first 100 words against 


TTR of last 100 words r= .978 
(2) Mean 100-word TTR of first 

500 words against mean 100- 

word TTR of last 500 words. r= .669 
(3) Mean 100-word TTR of first 

1,500 words against mean 100- 

word TTR of last 1,500 words. r= .826 
(4) First 500-word TTR against 

last 500-word TTR. r = .657 


(5) Mean 500-word TTR for first 
,500 words against mean 500- 
word TTR of last 1,500 words. r= .829 

(6) First 1,000-word TTR against 
last 1,000-word TTR. 


If it is assumed that the various type- 
token ratios consideration are 
equivalent measures, we see that the 
1,000-word I’TR is practically as good a 
measure of the individual's language as 
the average 100 and 500-word TTR’s for 
1,500 Further, the 1,000-word 
TTR gives a reliability which compares 
favorably with that estimated for average 
100 and 500-word TTR’s based on the 
full 3,000 words, .813 as compared to 
.go4 and .go6. For most purposes, a type- 


under 


words. 


token ration based on 1,000 words will 
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prove adequate and in some instances 
[TR’s based on only 500 words might 
prove useful. 

Finally we seek to answer the ques- 
tion: ‘How strongly are the type-token 
ratios interrelated?” An answer to this 
question will also give a partial answer 
to the question: “Do the different type- 
token ratios employed in this study 
measure the same aspect of language?”’ 
An answer to these questions was sought 
by intercorrelating the four type-token 
ratios (all four based on 3,000 words). 
These correlation coefficients are as fol- 
lows: 


R 100 Rs R,, 000 
R 500 .934 
R000 .870 .948 
R33, 000 -745 925 O52 


The multiple-correlation coefficient of 
R.oor Rsoor ANd Ry ooo With Ry ooo 18 -99- 
Besides the fact that the four T’TR’s are 
highly interrelated it is noted that the 
greater the difference between the base 
number of words the less the correlation. 
Since, for each individual, these T'TR’s 
are based on the same language sample, 
this result is to be anticipated, to some 
extent, on a priori grounds. Further, in- 
spection of the scatter-diagrams of these 
intercorrelations revealed that the rela- 
tionships are linear. On these grounds, 
we would judge the four T’TR’s to meas- 
ure essentially the same aspect of lan- 
guage. 


GROUP DIFFERENCES IN LANGUAGE MEASURES 


Differences between the levels of I.Q., 
C.A., locality and sex were investigated 
by means of the analysis of variance 
technique. The analysis of the factorial 
design was carried out for the following 
language measures: 


1. Language measures derived from the 
total sample: 


a. Type-token ratio® for 100-word seg- 
ments, 

b. Type-token ratio for 500-word seg- 
ments. 

c. Type-token ratio for 1,000-word seg- 
ments. 

d. Type-token ratio for the total 3,000 
words. 

. Language measures derived from. sec- 
tions of the samples categorized by the 
following parts of speech: nouns, verbs, 
adjectives, and adverbs. Each of these 
four sections of the samples were ana- 
lyzed separately in terms of the follow- 
ing measures: 

a. Number of tokens. 

b. Number of types. 

c. Type-token ratio computed as the to- 
tal number of types divided by the 
total number of tokens of each cate- 
gory. 

d. The percentage which the types of 
each category is of the total number 
of types in the four categories. 


no 


From these measures, a total of twenty 
analyses of variance was made. In the 
tables presenting these results the fol- 
lowing symbols are used to designate the 
various levels of the factors under con- 
sideration: 


1. 1.Q. levels 
I, —1.Q. 89 and under. 
I,—1.Q. go to 109, inclusive. 
I,—1.Q. 110 and over. 
2. Chronological age levels 
A,—C.A. of 12 years, five months and 
under 
A,—C.A. of 12 years, six months to 
14 years, 11 months, inclusive. 
A,—C.A. of 15 years and over. 
3. Locality levels 
L,—City 
L,—Town 
L,—Rural 
4. Sex 
S,—Boys 
S,—Girls 


In the twenty applications of the anal- 
ysis of variance technique, 208 interac- 


* All type-token ratios are presented as per- 
centages rather than as decimal fractions. 
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TABLE 5 
Summary of results of analysis of variance for 20 language measures 























































1.Q. a ® Locality Sex 
Level of Rank order* Levelof Rankorder Levelof Rankorder Levelof Rank order 
signif. of means signif. of means signif. of means signif. of means 
F-test of groups F-test of groups F-test of groups F-test of groups 
Ri 1% IilsIs 1% AiAsAs 
Roo 1% Ile; 1% AiA2As 
Rio I % Iilels I % AiA:As 
Raooo I % lial; I % AiA2As; 
Number of Tokens 
Nouns 5% IilsIs 1% LeLsLi b 2 
Verbs 1% LiLsL: 5% Boys Girls 
Adjectives ; 
Adverbs 5% LiLsL2 5% Boys Girls 
Number of Types 
Nouns 1% IikcIs 
Verbs 5 % Iil-Is 
Adjectives 5% IiIels 5% AiA2As 
Adverbs I % AiAsA2 
Type-Token Ratio for 
Nouns 1% IiIels 5% AiA2As 
Verbs 5% Iilal; 5% LiLsLi 
Adjectives 5% IIeIs 5% AiAaAs 
Adverbs 5% Iilels 5% LeLsLi 
Per cent of Total Types 
Nouns 
Verbs 5 % IsIoty 
Adjectives 1% AiA,As 
Adverbs 1% IsIeI; 


5% LiLeLs 

















Li—City; L:-—Town; L:—Rural. 


; tion variances were computed, and since 
only three of these were significant, and 
then only at the five per cent level of 
confidence, it was felt that, on the 
grounds that these measures are highly 
interrelated, it could be safely assumed 
that there is no interaction among the 
four factors under consideration as far 
¥ as these language measures are con- 
dh cerned. By chance one should expect 
about ten or eleven of the interaction 
variances to be significant at the five per 
cent level of confidence. 





Because the main purposes of this 
analysis were exploratory, it was decided 
that the more conservative error variance 
estimate should be used to test the main 
effects. In 19 of the 20 individual anal- 
yses, the Ix Ax LxS§ variance afforded 
the more conservative estimate of the 
error variance and consequently was 
used as the error term even though the 
degrees of freedom available for the F- 
test were much reduced. In the one in- 
stance where the Ix Ax LxS variance 













* Rank order of means in terms of increasing magnitude. ; 
Legend: I:—1.Q. 89 and under; I.—I.Q. 90 to 109 inclusive; I:—I.Q. 110 and over. ; 
: 1—C.A. 12-5 years and under; A:—C.A. 12-5 to 14-11 years inclusive; Asx—C.A. 15 years and over. 





was less than the majority of the inter- 
action variances, an error variance was 
computed by summing all of the sums 
of squares of the interaction terms and 
dividing by the sum of the degrees of 


‘ freedom of all of the interaction terms. 


In cases where the F-test was signifi- 
cant, differences between levels were 
tested by means of Fisher’s t-test. ‘The 
error variance used as a basis for these 
t-tests was computed as the residual vari- 
ance after the variation due to the main 
effects had been deducted from the total 
variance. This procedure is permissible 
on the hypothesis of no interaction and 
since this hypothesis of no interaction 
seems tenable, the standard error of dif- 
ference was computed from these resid- 
uals because it permitted a greater num- 
ber of degrees of freedom in evaluating ¢. 


Summary of Results of Analysis of 
Variance for Each of 20 Language 
Variables 


The evidence garnered from the ap- 
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plication of the analysis of variance tech- 
nique to 20 language variables, reveals 
their capacity to differentiate groups 
classified according to the factors inves- 
tigated. A summary in tabular form is 
presented in Table 5.4 Considering the 
fact that the results of the analyses of 
variance of these 20 variables are rather 
ponderous, a. skeletonized version of 
these results would appear to be more 
appropriate than a detailed account. 
The summary will collect the significant 
results for each of the pertinent factors. 

A. The I.Q. factor. Of the 20 analyses 
of variance involving the I1.Q. factor 14 
resulted in significant F-values. Of these, 
seven are significant at the one per cent 
level and seven at the five per cent level 
of confidence. In general, the direction 
of differences of the means of I.Q. levels 
for these variables is in a numerical in- 
crease in the value of the measure for 
increases in I.Q. level. The I.Q. factor 
seems to be the most strongly related to 
these language measures. 

1. Segmental type-token ratios 

The 100, 500, 1,000 and 3,000-word 
type-token ratios all give F-values sig- 
nificant at the one per cent level of con- 
fidence. Means of the three I.Q. levels 
for the segmental type-token ratios are 
positively related to I.Q. level. 

2. Variables dependent on counts of 
nouns 

Three of the four measures derived 
from counts of nounal types and tokens 
resulted in significant F-ratios. ‘The type- 
token ratio and number of types, respec- 
tively, are significant at the one per cent 
level of confidence, while the percentage 
of nounal types failed to reach either 
criterion of significance. In each case, 
differences in means of these measures 





*A complete presentation and discussion of 
the results of the analysis of variance are on file 
at the $.U.I. library. 


among I.Q. levels is in favor of an in- 
crease in the mean value of the measure 
with an increase in I.Q. level. 

3. Variables dependent upon a count 
of verbs 

Three of the four measures involving 
counts of verbal types and tokens result 
in significant F-values, all significant at 
the five per cent level of confidence. 
These variables gave significant F-values: 
number of verbal types, type-token ratio 
for verbs, and percentage of verbal types. 
The direction of differences among 
means of I.Q. levels for the number of 
types and type-token ratio, respectively, 
is positively related to I.Q. levels, while 
the direction of mean differences for the 
percentage of verbal types is reversed, 
the low I.Q. group using a greater per- 
centage of verbal types than either of 
the other two I.Q. groups. 

4. Variables dependent upon a count 
of adjectives 

Measures based on a count of the num- 
ber of adjectival types and tokens result 
in only two of the four measures giving 
significant F-ratios for the I.Q. factor. 
Both of these are significant at the five 
per cent level of confidence. The direc- 
tion of mean differences for these two 
significant measures is positively related 
to I.Q. level. 

5. Variables dependent upon a count 
of adverbs 

Of the four measures involving ad- 
verbs, the type-token ratio for adverbs 
and the percentage of adverbal types 
gave significant F-ratios, the latter sig- 
nificant at the one per cent level of con- 
fidence and the former at the five per 
cent level. I.Q. levels prove to be posi- 
tively related to the mean _ type-token 
ratio for adverbs and negatively related 
to the mean percentage of adverbal 
types. 

B. The C.A. factor. Nine of the 20 
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language measures, when each is sub- 
mitted to an analysis of variance in terms 
of the factorial design, result in a sig- 
nificant F-value for the C.A. factor. Of 
these nine measures, six give F-values 
significant at the one per cent level and 
the remaining three give F-values sig- 
nificant at the five per cent level of con- 
fidence. For each of these nine measures, 
the means of C.A. levels increase in value 
with an increment in C.A. level; in other 
words, the older the child the higher the 
score in terms of these nine measures. 

1. Segmental type-token ratios 

The four segmental type-token ratios, 
when submitted to an analysis of vari- 
ance, result in F-values which, for the 
C.A. factor, are significant at the one 
per cent level of confidence in each case. 
The direction of mean differences is posi- 
tive, i.e., the older the child the higher 
the numerical value of the type-token 
ratios. 

2. Variables dependent upon a count 
of nouns 

Only the type-token ratio for nouns 
resulted in a significant F-value, and this 
at the five per cent level of confidence. 
The older children tended to have a 
greater type-token ratio for nouns. 

3. Variables dependent upon a count 
of verbs 

None of the four variables derived 
from counts of verbal types and tokens 
results in significant F-values for the 
C.A. factor. 

4. Variables dependent upon a count 
of adjectives 

Three of the four variables involving 
counts of adjectives prove significant for 
the C.A. factor. For this factor, the num- 
ber of adjectival types is significant at 
the one per cent level. This evidence 
points to an increase in the use of ad- 
jectival tokens and types as the children 
grow older. 
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5. Variables dependent upon a count 
of adverbs 

Of the variables in this category only 
the number of adverbal types gives a 
significant F-value (at the one per cent 
level) for the C.A. factor in the analysis 
of variance. The tendency is for an in- 
crease in the use of adverbal types with 
age, although the two older groups are 
reversed, but the difference in means be- 
tween these two groups is not statistically 
significant. 

C. The locality factor. Six of the 20 
measures, when submitted to an analysis 
of variance, gave significant F-ratios for 
the locality factor. Two of these six are 
significant at the one per cent level and 
the remaining four at the five per cent 
level of confidence. No general trend in 
differences among the means of the city, 
town and rural groups was noted. 

1. Segmental type-token ratios 

Segmental type-token ratios do not 
differentiate locality groups. None of 
these four measures gave significant F- 
ratios for the locality factor. 

2. Variables dependent upon a count 
of nouns 

For the locality factor, only one meas- 
ure, number of nounal tokens, gave a sig- 
nificant F-value (at the one per cent 
level). The city group uses, on the aver- 
age, a greater number of nouns than do 
town or rural groups, while the rural 
group uses more than do the town group. 

3. Variables dependent upon a count 
of verbs 

For the locality factor, the number of 
verbal tokens and type-token ratio for 
verbs, respectively, gave significant F- 
values. Only the number of verbal tokens 
is significant at the one per cent level of 
confidence. The rank order of means for 
number of verbal types is town, city and 
rural in order of decreasing magnitude, 
while for the type-token ratio for verbs 
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he corresponding rank order is city, 
rural and town. 

:. Variables dependent upon a count 
ol adjectives 

For the locality factor, measures based 
on counts of adjectival types and tokens 
do not produce any significant F-values. 

5. Variables dependent upon a count 
of adverbs 

Three of the four variables derived 
from counts of adverbal types and tokens 
successfully differentiate locality groups 
as judged by the F-test. These three 
measures are the number of adverbal 
tokens, adverbal type-token ratio, and 
percentage of adverbal types. All three 
are significant at the five per cent level 
of confidence. City groups tend to use 
the least number of adverbal tokens and 
types, but have the greatest adverbal 
type-token ratio. The town group uses 
the most adverbal tokens while the rural 
group uses a greater percentage of ad- 
verbal types. 

D. The sex factor. For the sex factor, 
only two of the go language variables 
gave significant results in terms of the 
analysis of variance. These two measures 
are number of verbal tokens and num- 
ber of adverbal tokens, respectively, both 
significant at the five per cent level of 
confidence. In both instances girls use a 
greater number of these classes of tokens 
than do boys. 

E. General summary. In general, it 
may be said that in terms of the lan- 
guage measures employed, the higher the 
1.Q. and the higher the age level the 
more highly differentiated is the lan- 
guage structure of the writers. The use 
of a proportionately greater number of 
nouns and adjectives characterizes high 
1.Q. and older age groups, while the use 
of a proportionately greater number of 
verbs characterizes the low I.Q. and 
younger age groups. Adverb usage is not 


clearly differentiating among __ these 
groups. 

On the basis of the analysis of variance 
one would predict that a correlation 
exists between the type-token ratios and 
I.Q. score and between type-token ratios 
and C.A. The correlation of type-token 
ratios with I.Q. scores might be attenu- 
ated by allowing C.A. to be unrestricted 
and on the other hand the correlation 
between C.A. and_ type-token ratios 
might be attenuated if I.Q. is allowed 
an unrestricted range due to the possible 
counteracting influence of these two fac- 
tors, although there might be reinforce- 
ment rather than attenuation. In any 
event, it is desirable to determine the 
relationship between I.Q. and type-token 
ratio with the effect of C.A. reduced to 
a minimum, and to determine the rela- 
tionship between C.A. and _ type-token 
ratio with the influence of I.Q. mini- 
mized. One method of accomplishing 
this end is to determine the correlation 
of I1.Q. with type-token ratio within 
C.A. levels and of C.A. with type-token 
ratio within I.Q. levels. ‘These correla- 
tions may be viewed as empirically de- 
termined partial correlation coefficients, 
in the first case with C.A, held constant 
and in the second with I.Q. held con- 
stant. In each case three measures of the 
partial correlation coefficient are ob- 
tained. 

On the basis of the assumed equiv- 
alence of the four type-token ratios being 
studied, only the correlations of the 
3,000-word type-token ratio with C.A. 
and I.Q. were computed. The correla- 
tion between 3,000-word type-token ratio 
and I.Q. and between 3,000-word type- 
token ratio and C.A. within I.Q. and 
C.A. groups was also computed.® These 


*See Lindquist, E. F. Statistical Analysis in 
Educational Research, Houghton Mifflin Co., 
1940, pp. 219-228. 
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TABLE 6 


Table of correlations of 3,000-word type-token 
ratio with C.A. and I.Q. 


Correlation of r N 
R;,000 With C.A. 332 108 
(1.Q. unrestricted) 
R;,000 With C.A. Within I, .463 36 
R3,000 With C.A. Within I, 395 36 
R;,000 With C.A. Within I; .463 36 
R3,o00 With I.Q. .517 108 
(C.A. unrestricted) 
R;,000 With 1.0. Within A, .646 36 
R3,000 With 1.0. Within Az -552 36 
R3,o00 With 1.Q. Within As; —_. 548 36 
Within Groups Correlation of 
R3,000 With C.A. 325 108 
R3,000 With 1.Q. .420 108 





correlations are presented in Table 6. 
In each case the unrestricted correlation 
along with the correlation within groups 
as well as the empirically determined 
partial correlations are presented. A 
comparison of these two correlations re- 
veals that the unrestricted correlation 
and the within groups correlations tend 
to be very much alike in this instance. 
This result is to be expected since the 
range in C.A. and I.Q. is reduced rela- 
tively much more than is the range of 
the type-token ratio scores. It would 
seem that a better index of the strength 
of the felationship of the 3,000-word 
type-token ratio to I.Q. and to C.A, is 
the correlation of 3,000-word type-token 
ratio with I.Q. within C.A. levels on one 
hand ahd 3,000-word type-token ratio 
with C.A. within 1.Q. levels on the other, 
since in each case counteracting influ- 
ences are somewhat reduced. However, 
inasmuch as the locality and sex factors 
were not significant, the within groups 
correlations permits the estimation of 
the significance of the correlation of I.Q. 
and of C.A. with 3,000-word type-token 
ratio. With 100 degrees of freedom, both 
correlations are significant at the one 
per cent level of confidence when tested 


by means of Fisher’s t-test. The t-value 
for the within groups correlation of C.A. 
with 3,000-word type-token ratio is 3.51; 
the equivalent measure for the within 
groups correlation of I.Q. with 3,000- 
word type-token ratio is 4.64. 

The positive results of these correla 
tions are indicative of a relationship be- 
tween type-token ratios and I.Q. and 
between type-token ratios and C.A., but 
the relationships are not strong enough 
to predict a type-token ratio for an in- 
dividual from knowledge of his age and 
his I.Q. score. On the assumption that 
the correlation of C.A. and I.Q. is zero, 
the multiple correlation coefficient of 
type-token ratio with C.A. and I.Q. is 
only .608, a result which suggests that 
C.A. and 1.Q. are not sufficient factors 
for completely determining the type- 
token ratio. 


CUMULATIVE TYPE FREQUENCY CURVE 


In a recent article, Carroll (3) states 
that the equation 


Z 


D — — (.423 + K — log.N + log, (1) 
K 

where D is the number of different words 
in a language sample of length N, N is 
the total number of words in that sam- 
ple, and K is an empirically determined 
constant, held for the language samples 
he had under investigation. If this for- 
mula could be demonstrated to hold 
generally, it would be a powerful tool in 
language research. 

Carroll deduced equation (1) by means 
of certain logical and statistical consid- 
erations from the following equation, 

N 
r= (2) 
KR* 
where F is the frequency with which any 
given word occurs, R is its rank in order 


*See Lindquist, E. F. Op. cit., pp. 210-211. 
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LOG F = -0.850 LOG R + 2.437 
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Fic. 1.—Subject no. 12. Graphic representation of the relationship of the rank of a word (R) in 
decreasing frequency order to the frequency of occurrence (F). The upper plot shows the reduc- 
tion line in terms of log F and log R. Empirical points are shown in their relation to the curve 


described by the indicated equation. 


of decreasing frequency, N the number 
of words in the sample from which F 
and R are computed, and K is an em- 
pirically determined constant which has 
the same meaning as the K in equation 
(1). A necessary condition to the applica- 


bility of equation (1) is that equation 
(2) hold for the data and particularly, 
that the exponent of R have a value of 
1.0. 

To determine whether equation (2) 
holds for the language samples under 
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investigation, 18 of the 108 language 
samples were selected in such a fashion 
that two randomly selected cases came 
from each of nine C.A., I.Q. groups. ‘The 


TABLE 7 
Estimates of parameters, a and K in the fitted 
equation F =3,o00/KR® for eighteen 
language samples 








First Twenty 

















Subject Total Ranks nso , 
a iminate 
a K a K 
20 0.938 6.716 1.255 2.028 
49 0.825 10.549 1.021 4.636 
79 0.865 9.080 I .000 5.296 
3 0.859 9.709 0.995 5-625 
96 0.855 10.211 ©.99I 5-931 
78 ©.796 II.700 1.041 4.636 
8 ©.916 8.513 0.929 8.378 
4 0.850 10.989 ©.937 7.382 
56 0.817 11.646 ©.949 6.701 
12 0.828 11.619 0.948 6.969 
IOI 0.852 10.142 1.049 4.370 
86 0.815 12.948 °.889 9.204 
44 0.825 11.891 ©.94I 7.246 
97 0.827 10.741 1.065 3.909 
43 0.809 14.4905 0.808 14.528 
6 o.861 10.003 1.010 5-372 
100 0.829 11.299 0.907 8.319 
22 0.841 12.448 0.850 12.165 
Means 0.845 0.974 





variables F and R were measured for 
each sample. Equation (2) can be re- 
duced to 
N 
Log F = —a log R + log = 


which is seen to be linear in log F and 
log R. If a plot of (log F, log R) can be 
considered linear, then the line of best 
fit can be determined by means of the 
method of least squares. A graphical rep- 
resentation of a representative plot of 
(log F, log R) and of (F, R) along with 
the best fitting curve is presented in Fig- 
ure 1. Similar curves were computed for 
each of the 18 selected samples of lan- 
guage. It can be noted that the fit is not 
good for the lower ranks, and since Car- 


roll states that equation (2) holds only 
for ranks greater than about 20, the best 
fitting straight line was fitted to each of 
the 18 plots of (log F, log R) for a series 
of points in which the first 20 (approxi- 
mately) points corresponding to the 
lower 20 ranks were eliminated. Esti- 
mates of the parameters, a and K, for 
18 language samples are presented in 
Table 7. 

If equation (1) is to have any general- 
ity, then equation (2), which as was 
noted previously is the harmonic series 
law of word frequency distribution, must 
hold for language samples in general. 
Specifically it must be demonstrated that 
a plot of (log F, log R) is linear and that 
the value of the exponent of R is 1.0. 
On the basis of the 18 plots of values of 
(log F, log R), it was judged that the as- 
sumption of linearity is reasonable, al- 
though the possibility of some other 
function giving a better fitting reduction 
line should not be excluded. As for the 
exponent of R, it can be noted that the 
curves in which the first 20 ranks were 
eliminated result in values of a which 
are much closer to 1.0 than are the values 
of a for the entire series of ranks. Fur- 
ther, we note that when the first 20 ranks 
are eliminated, several of the 18 equa- 
tions give estimates of a which are prac- 
tically 1.0. On the other hand we note 
quite a range in the a values, from 0.808 
to 1.235, and we are confronted with the 
problem of determining if these esti- 
mates of a are sufficiently close to 1.0 to 
support the assumption that the value 
of this parameter is 1.0. 

On the assumption that the value of 
the parameter a is 1.0, we can, by means 
of the t-test test the hypothesis that the 
mean value of a for these 18 language 
samples differs from 1.0 within the limits 
of chance. The mean of the 18 a-values 
differs from 1.000 by 0.026 and results 
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TABLE 8 


Table of K-values for eighteen language samples computed at successive 
five-hundred-word points 











Subject 500 1,000 1,500 








2,000 2,500 3,000 
Number Words Words Words Words Words Words Mean 
79 5.69 6.03 6.24 6.45 6.56 6.56 6.24 
3 6.49 6.67 6.81 6.86 6.99 7.10 6.82 
96 6.94 6.97 7.14 7.22 7.28 7.28 7.14 
6 7.14 7.08 7.22 7.29 7.30 7.30 7.22 
78 7.13 6.88 6.82 6.88 6.91 6.99 6.94 
49 6.50 6.75 6.87 6.87 6.90 6.94 6.81 
IOI 6.66 6.65 6.82 6.96 7.09 7.13 6.89 
56 6.90 7.15 7.15 7.28 7.31 9.33 7.19 
12 6.59 6.78 7.00 7.17 7.42 7.41 7.06 
44 7-47 7.38 7.46 7-39 7-31 7-41 7.40 
4 6.92 7.83 7.20 7.27 7.36 7.36 7.24 
97 6.88 6.89 6.84 6.87 6.90 6.98 6.89 
8 7.08 6.75 6.93 6.99 7.06 7.13 6.99 
100 7-43 7.24 7.25 7.36 7-40 7-43 7-35 
86 7-47 7-45 7-47 7-57 7-53 7.62 7-52 
22 . “5 7 7.60 7.81 7.85 7.92 7.65 
43 7-91 8.02 7.80 7-93 7-97 8.05 7-95 
29 6.17 6.22 6.44 6.48 6.58 6.68 6.43 
Means 6.928 6.972 7.059 7.153 7.207 7.256 





in a t-ratio of 1.178, which, with 17 de- 
grees of freedom, gives a_ probability 
value greater than 0.2 but less than 0.3, 
a probability interval which, judged by 
the ordinary criterion of significance, is 
not significant. This statistical test is not 
entirely satisfactory, however, since the 
assumption we wish to test with refer- 
ence to the value of the parameter a is 
that it is 1.0 in each individual case and 
not that the mean of a distribution of 
randomly selected language samples is 
1.0, although the latter is necessarily true 
if the former holds. On the basis of this 
test, it is reasonable to accept the hy- 
pothesis that the mean value of the pa- 
rameter a might be 1.0, and, in some de- 
gree, the assumption of a harmonic series 
law of word frequency distribution is 
seen to be validated when approximately 
the first 20 ranks have been eliminated. 

Equation (1) is peculiar in that it 
contains only one parameter, K. An 
estimate of this parameter can be de- 
termined from any one point on the 
cumulative type-frequency curve. If K 


is constant, as it must be if equation (1) 
is to hold, then estimates of K computed 
at various points along the cumulative 
type-frequency curve should differ from 
one another in a chance fashion. Since 
we can get a distribution of K-values for 
each individual sample, as well as a dis- 
tribution of K-values at each successive 
point along the curve for a group of in- 
dividuals, we can derive two estimates of 
the population variance, one» based on 
the variance due to the difference in 
means at successive points and the other 
a remainder variance computed from the 
total sums of squares after the variation 
due to individuals and to successive 
points along the curve have been de- 
ducted. On the assumption that the vari- 
ous estimates of K differ only by chance, 
the F-ratio of the two estimated variances 
should be non-significant. 

For the same 18 language samples used 
to test equation (2), K-values were com- 
puted at each successive 500-word point 
along the cumulative type-frequency 
curve. Mean K-values for the 18 samples 
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TABLE 9 
Results of analysis of variance of K-values for eighteen language samples 














Factor Sums of Squares d.f. Variance F 
Individuals 17.0892 17 
Sample size 1.5524 5 ©.3105 18.356 
Error 1.4377 85 ©.0169 

Total 20.0793 107 





Inter-mean K-value differences 





500 Words 1,000 Words’ 1,500 Words 2,000 Words 2,500 Words 
1,000 Words 0.044* 
1,500 Words 0.131 0.087 
2,000 Words 0.225 0.181 0.094 
2,500 Words 0.279 0.235 °.148 0.054 
3,000 Words 0.328 0.284 0.193 0.102 0.049 





* Differences greater than 0.0848 are significant at the 5% level of confidence. Differences greater 
than 0.1116 are significant at the 1% level of confidence. 


were also computed at each successive 
500-word point as well as the mean K- 
value for each sample. ‘These data are 
presented in Table 8. It is noted that 
there appears to be a systematic tendency 
for the value of K to increase as the base 
number of words from which it is com- 
puted increases, although this tendency 
is not of equal strength in all cases.’ 
The data in Table 8 were subjected 
to an analysis of variance in order to 
determine if the variation in mean K- 
values at successive 500-word points 
could, statistically, be allocated to chance 
factors. The error variance used to test 
the sample size variance, i.e., variation 
derived from the means at successive 
500-word points, was the interaction 
variance of individuals and sample size. 
The results of this analysis of variance 
are presented in Table g. The F-ratio 
of 18.356 is significant at the one per 
cent level of confidence. This result may 
be interpreted as meaning that the mean 
K-values computed at successive 500- 
word points along the curve cannot be 


‘In one case, subject number 79, the value of 
D at N = 3,000 was not large enough for a valid 
computation of K. In order to complete the de- 
sign the value of K computed at 2,500 words was 
used as the best estimate of K at N = 3,000. 


considered as representing populations 
which are equally variable or which have 
equal means. On the basis of a signifi- 
cant F-test, inter-mean K-values were 
tested by means of the t-test. Of the 15 
differences among the six means, 12 give 
a t-value significant at least at the five 
per cent level and nine give t-values sig- 
nificant at the one per cent level. If the 
means of the K-values differed by chance 
only, one would expect less than one of 
the 15, differences to be significant at the 
five per cent level when tested by means 
of the t-test. 

A further test of the adequacy of the 
hypothesis that K is constant is afforded 
by an analysis of the behavior of K in 
the 18,000-word sample. K-values were 
computed at successive 1,000-word points 
throughout the 18,000 words. Since K 
cannot be validly estimated at very large 
values of N it was necessary to determine 
whether or not K could be validly com- 
puted at each of the successive 1,000- 
word points along this curve. In equa- 
tion (1), D is a double-valued function 
of N, ie., there are two values of N 
which will satisfy the equation for any 
given D. Disregarding the portion of the 
curve described by equation (1) for nega- 
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tive values of N, the curve may be said 
to have its origin at point (0.0), to rise 
to a maximum and to fall indefinitely 
for all values of N beyond this maxi- 
mum. The usable portion of the curve 
is from its origin to its maximum, The 
maximum point on the curve is deter- 
mined from K and for each value of K 
a maximum WN can be computed beyond 
which the value of D computed from 
equation (1) is not valid. If for each K 
there is a maximum N, then for each 
value of N there is a minimum value of 
K which is valid for that value of N. 
These minimum values of K for specified 
N’s can be determined by setting the first 
derivative of equation (1) equal to zero 
and solving for K. The first derivative of 
equation (1) is 

dD 1 


— ——(K +4 log, K — log. N — 0.577) (4) 
dN K 


and setting 


dD 

—=-=0 

dN 

Log. N = K + log, K — 0.577 (5) 


as the equation from which we can de- 
termine the maximum point on the 
curve for specified values of N. In terms 
of the independent variable, N, the lim- 
its of the usable portion of the curve 
derived from equation (1) are from 
N =o to 


N= Ee, log. K - 0.577)+ (6) 


From equation (5) minimum values of 
K, which for a specified N give a maxi- 
mum, can be solved. From this value of 
K, then, the minimum D value for the 
specified N and K can be computed from 
equation (1). These minimum D and K 
values, along with the empirically deter- 
mined K-values at each successive 1,000- 
word point of the cumulative type-fre- 
quency curve, are presented in Table 10. 

Apparently K-values can be validly 


TABLE 10 


K-values computed at successive 1,000-word 
points of the 18,000 word sample 








Minimum 
. Minimum D for 
N D K K for which K 
given N is valid 





1,000 434 7.82 5-73 175 
2,000 732 8.03 6.35 316 
3,000 936 8.00 6.68 449 
4,000 1,175 8.18 6.94 576 
5,000 1,376 8.27 7.13 701 
6,000 1,532 8.27 7.20 823 
7,000 1,700 8.34 7-42 943 
8,000 1,889 8.42 7.54 1,061 
9,000 2,021 8.45 7.64 1,178 
10,000 2,193 8.51 7.74 I,292 
11,000 2,344 8.56 7.83 1,405 
12,000 2,465 8.58 7.90 1,519 
13,000 2,585 8.62 7.97 1,631 
14,000 2,703 8.64 8.04 1,741 
15,000 2,786 8.64 8.10 1,852 
16,000 2,884 8.66 8.16 1,961 
17,000 2,966 8.66 8.21 2,071 
18,000 3,025 8.67 8.26 2,179 





computed throughout the length of this 
sample. However, it is to be noted that 
if we were making a prediction of D 
from 3,000 words, the number of words 
collected from the other subjects, the 
estimate of the maximum number of 
different words in this child’s vocabulary 
would fall somewhere between 2,585 and 
2,703 words, i.e., at the point along the 
N-axis beyond which computation of D 
from K = 8.00 is no longer valid. ‘The 
data do not justify the prediction that 
the child’s vocabulary is exhausted after 
writing between 13,000 and _ 14,000 
words, as new words are being added 
throughout the length of the sample. 

On the other hand, it is noted that 
beyond N equal to about 11,000 or 
12,000 the value of K remains fairly 
stable. This sample, like the previously 
discussed samples, gives estimates of K 
which show a systematic tendency to in- 
crease for successive values of N. 

The evidence points to a rejection of 
the hypothesis that, for these language 
samples, K is a constant. Indirectly, it 
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LOG D = 0.713 LOG N + 0.392 
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.—A graphic representation of the relationship of the number of different words (D) to the 
sample size (N). The upper curve is the reduction plot in terms of log D and log N. Plotted 
from the data computed from language sample written by subject no. 96. Empirical points are shown 
in their relation to the curve described by the indicated equation. 








= g means that equation (1) will not ade- samples like those used in this investiga- 
fF quately describe the data derived from tion, with the qualification that 3,000 
aa these samples and that D will vary sys- words may be an insufficient number of 
re. tematically from the obtained values of | words on which to base an estimate of K 
a as D. We may, on this evidence, reject the for prediction of D’s. 

Ff generality of equation (1) for language An attempt was made to find an em- 
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TABLE 11 


Estimates of parameters, a and b, in the fitted 
function D=bN® for 18 language samples 


TABLE 12 


Results of analysis of variance of parameter a for 
groups of I.Q., C.A. and type-token ratio 








Subject 





Number ° b 
29 0.618 3-373 
49 0.657 3.192 
79 o.702 1.656 

3 0.725 1.903 
96 0.713 2.466 
78 ©.600 5.023 

8 0.646 3-733 

4 0.740 2.109 
56 ©o.710 2.570 
12 0.743 1.945 

IOI 0.683 2.754 

86 0.722 2.606 
44 0.690 3.170 
97 0.599 4-989 
43 0.729 2.786 

6 0.679 3-221 

100 0.665 3-750 

22 0.704 1.614 





pirically fitted equation to represent the 
cumulative type-frequency curves. Since 
it is reasonable to assume that, for any 
individual, there is a limit to the num- 
ber of types at his command, it was felt 
that an equation of the hyperbolic form 
would best agree with the character of 
the phenomena at hand, inasmuch as the 
hyperbolic curves are characterized by 
asymptotes, which can be correlated to 
the limit of the writing vocabulary of 
the individuals in similar situations. 
However, it was found that the data 
could not be satisfactorily reduced to a 
linear form of the hyperbolic curve, at 
least not a simple hyperbolic curve with 
two parameters. Possible reasons for this 
failure will be discussed later. 

Of the attempts to fit curves with sim- 
ple equations to these data, only a plot 
of log D and log N resulted in what 
could be considered a linear relation- 
ship. The resulting linear function is of 
the form 

log D=a log N+ log b 


in which a and b are empirically deter- 
mined constants. If the above equation 





Sums of df. 


é r Variance F 
Facto Squares 





1.Q. 0.004449 2 0.002225 1.642 
C.A. ©.O10095 2 0.005048 3-723 
TTR ~~ 0.015547 I 0.015547 #2«311.474* 
Error 0.016261 I2 0.001355 


Total 0.046352 17 











Factor Mean _ Difference t 
I.Q. 
89 and under 0.669 
go to 109 (inc.) 0.707 
110 and over 0.693 
CA. 


149 mo. and under 0.656 
150 to 179 mo. 

(inc.) 0.709 
180 mo. and over 0.704 


TTR for 3,000 words 
Less than .231 0.660 
Greater than .231 0.719 0.059 3-411* 





* Significant at the one per cent level of confi- 
dence. 
holds, then log D is a linear function of 
log N, and D is a power function of N, 
of the general form 

D = bN*. 

Equations of this form were fitted to the 
cumulative type-frequency data for the 
18 language samples used in fitting equa- 
tion (2). Estimates of parameters a and b 
arrived at by means of a least squares 
solution of the (log N, log D) plot, are 
presented in Table 11. A graphical rep- 
resentation of this relationship for a 
typical plot of (log N, log D) and (N, D) 
is shown in Figure 2. The curve pre- 
sented in Figure 2, is typical of the other 
fitted curves in that the fit for larger 
values of N is not satisfactory and would 
make prediction beyond the limits of 
these data rather hazardous. 

In order to determine what relation- 
ship exists between the estimates of these 
parameters and the factors of I.Q., C.A. 
and 3,000-word segmental type-token ra- 











) 
| 
) 
: 
. 
: 
) 
' 

















ee 














102 


TABLE 13 
Results of analysis of variance of estimates of 
parameter b for groups of I.Q., C.A., and 
type-token ratio 
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TABLE 14 
Estimate of parameters, a and b in the equation 
D=bWN? for 3,000-word sections and for the 
total sample of the 18,000-word 
language sample 


























~ Sums of ros SS ee = 
Factor Squares if. Variance F Sample Section a b 
1.Q. 1.211853 2 0.605927 — Total 18,000 Words 0.697 3-499 
C.A. 1.220563 2 0.610284 — 
TTR 1.606827 I 1.606827 — First 3,000 Words 0.729 2.825 
Error 12.739341 12 1.061612 Second 3,000 Words 0.752 2.239 
ee Third 3,000 Words 0.785 1.694 
16.778584 17 Fourth 3,000 Words 0.752 2.193 
—_ Fifth 3,000 Words 0.712 2.884 
Factor Mean Sixth 3,000 Words 0.680 3.236 
I.Q. 
89 and under 2.946 : , a , 
90 to 100, inclusive ey tween means of the various groups that 
110 and over 3-255 are of great enough magnitude to give 
CA. significant F-values. Since these factors 
149 months and under 3-279 have been shown to be associated in sys- 
150 to 179 months, inclusive 2.646 , ‘ 
odin tsmitine aah ever 2.895 tematic manner to the 3,000-word seg- 
mental type-token ratio the marked dif- 
TTR for 3,000 words f hisntiiier cif 1 Dita aol 
0.231 and less 3.239 erences between the two lower levels o 
©.232 and over 2.641 





tio, three levels each of 1.Q. and C.A., as 
previously defined and two groups, a 
low and a high, categorized according 
to the magnitude of the 3,000-word seg- 
mental type-token ratio, were subjected 
to an analysis of variance. The results 
of the analysis of variance of these two 
parameters for these groups is presented 
in Tables 12 and 13. The results of the 
analysis for parameter a indicates that 
only the type-token ratio factor results 
in a significant F-value. This result is to 
be expected since the type-token ratio 
is a function of D, one of the variables 
in the equation. Inasmuch as the con- 
stant a determines the rate at which new 
words are added, it is not surprising that 
the difference in means between the low 
and high 3,000-word TIR_ groups 
should result in a significant difference, 
when tested by means of the t-test, in 
favor of a greater magnitude in the 
value of a for the group with a larger 
type-token ratio. The I.Q. and C.A. fac- 
tors show no systematic differences be- 


I.Q. and C.A. might presumably be at- 
tributed to this association. 

The results of the analysis of variance 
of estimates of parameter Db for these 
groups indicate no significant factors. 
Apparently the exponent of N, 1.e., the 
parameter a, is more influential in deter- 
mining the differentiating characteristic 
of the curves than is the co-efficient of 
N, i.e., the constant b. 

In order to determine if the power 
function will hold beyond 3,000 words, 
a curve was fitted to the data of the 
18,000-word sample. The sample was di- 
vided into six 3,000-word samples and a 
curve fitted to each 3,000-word section 
as well as to the total sample. Estimates 
of the parameters of the power function 
for the total sample and for each section 
is presented in Table 14 and a graph- 
ical representation of the relationship 
for the 18,000 words is presented in Fig- 
ure 3. Again, it is noted that the fit for 
the larger values of N, beyond N equal 
about 14,000, is poor. The empirical 
points diverge considerably from the 
curve. 
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Fic. 3.—A graphic representation of the relationship of the number of different words (D) to the 
sample size (N) and of log D to log N for the 18,000-word language sample. Empirical points are 
shown in their relation to the curve described by the indicated equation. 


A second empirically fitted curve was 
derived by generalizing equation (1). A 
transformation of equation (1) can be 
made by writing the function as follows, 


log, N = -K(3) 4 (log. K + K + 0.423), (7) 


which, since the terms in the right-hand 
bracket are all constants can be written 
as, 


log, N= —K (2) + C. (8) 


Equation (8) is a more general function, 
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of which equation (7) is one special in- 
stance. In this form equation (8) is seen 
to be linear in (D/N, log, N). Thus, if a 
plot of the variables D/N and log. N can 
be considered as linear, we can proceed 
to make a least squares solution and get 
estimates of K and C. 

We note, further that if Carroll's for- 
mulation of the relationship, i.e., equa- 
tion (1), is to hold the two parameters of 
the fitted equation must have a definite 
relationship. Equation (1) makes the 
parameter C the following function of K, 
C = log, K + K + 0.423. (9) 
In this way we have a further test as to 
the adequacy of Carroll’s formulation, 
one that is more satisfactory since it uses 
all the data in the language sample 
rather than a few selected points along 
the cumulative type frequency curve. If 
equation (g) is found to be tenable, we 
have direct evidence in substantiation of 
equation (1); if not, we can substitute an 
expirically determined curve of the same 
type as equation (1) but of a more gen- 
eral nature, as follows: 

N 
D = — (C — log, N). (10) 
K 

In order to test the adequacy of equa- 
tion (8), an average value at each succes- 
sive 100-word point along the cumula- 
tive type-frequency curve was computed 
for each of the I.Q. levels of the experi- 
mental design. There were 36 language 
samples at each I.Q. level. An average 
series of points was felt desirable in order 
to give the empirical curve a greater 
stability at each point and also to smooth 
out chance fluctuations along the curve. 
The I.Q. levels were chosen because the 
variable D more clearly differentiated 
I.Q. levels than C.A., locality or sex 
levels. 


A graphical representation of the rela- 
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tionship between (D/N, log, N) and (D, 
N), as well as the best fitting curve for 
each of the three I.Q. levels, is presented 
in Figure 4. It can be seen from Figure 
4 that the reduction curves are reason- 
ably linear, and that the fit of the curves 
to the empirical points appears to be 
good. The empirically derived equations 
for these three curves are: 

(1) for the group composed of I.Q. 
89 and under, 


D 
log. N = —9.910 + 9.853 or 
N 
N 
D = ——_ (9.853 — log, N) 
9.910 


(2) for the group composed of I.Q. go 
to 109, inclusive, 


D 
log. N = —10.081 — + 10.272 or 
N 
N 
D — ———_- (10.272 — log, N) 
10.081 


(3) for the group composed of LQ. 
110 and over, 
D 
log. N = —9.551 — + 10.321 or 


N 
D— 





— (10.321 — log, N) 
9-55! 
K-values computed from equation (1) 
for these three curves at N = 3,000 are 
6.97, 7.24, and 7.40, respectively. Further, 
the restriction basic to equation (1), that 
C = K + log, K + 0.423 


does not appear to be fulfilled. On the 
basis of these results, we would be com- 
pelled to reject the generality of equa- 
tion (1) to these data. 

A plot of (D/N, log. N) was also made 
for the 18,000-word sample. The plot of 
(D/N, log. N) and D, N) as well as the 
best fitting curves in each instance, is 
presented graphically in Figure 5. Ap- 
parently the function described above 


represents the data reasonably well 
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throughout the 18,000 words. The em- 


pirically 


long sample is, 


log, 


D = 


N = —11.268 — + 11.670 or 


2m 


Fic. 4.—Graphic representation 
of the relationship of the num- 
ber of different words (D) to the 
language sample size (N) and of 
the ratio D/N to log.N. Plotted 
for each I.Q. level by averaging 
D at each successive 100-word 
point along the N-axis for each 
of the 36 subjects in that I.Q. 
level. Empirical points are shown 
in their relation to the curve 
described by the indicated equa- 
tion. 


Equations (1) and (10) have similar 


determined equation for this properties. Inasmuch as in equation (1) 


D 


N 
N 





(11.670 — log, N) 


the equivalent of parameter C is a func- 
tion of K, the maximum point on the 
curve is also a function of K. However, 
in the empirically determined curve, the 


11.268 -~-parameter C is independent of the con- 
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Fic. 5.— 


A graphic representation of the relationship of the number of different words (D) to the 


language sample size (N) and of the ratio D/N to log.N from the data computed from the 18,000- 
word language sample. Empirical points are shown in their relation to the curve described by the 


indicated equation. 


stant K, and it is noted that the maxi- 
mum point on the curve is a function of 
C and not K. For any given curve of this 
form, the maximum point on the curve 
is reached when 


¥(C -1) or 


If the value of N is greater than Ey _,), 


computations of D for these values of N 
are spurious. 
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V. DISCUSSION 


Crucial to the interpretation of the re- 
sults of this study is the manner of selec- 
tion and definition of a unit of language. 
\ny one of several language units might 
have been chosen—syllable, word, phrase, 
or clause, among others. The simplest 
and least ambiguous unit to define is the 
word, For this among other reasons, the 
unit of language used in this study is the 
word, and it is generally referred to as 
a token in order to differentiate it from 
a unit of vocabulary, the type. This defi- 
nition of a language unit is in conform- 
ance with that used by Zipf (15), Carroll 
(3), Fairbanks (5), Mann (9g), and Fos- 
sum (6) in studies which are comparable 
to this one. The word, as a unit of lan- 
guage, results in a statistical measure 
which involves fundamentally simple 
enumeration. All words are given equal 
weight in the determination of relation- 
ships. It may be questioned whether such 
a definition of language is entirely satis- 
factory, since it is known that a large 
proportion of words in connected dis- 
course must necessafily be made up of 
the structural or interstitial words which 
represent relationships among other 
words and, in and of themselves, carry 
no meaning independent of the imme- 
diate verbal context in which they ap- 
pear. On this basis, any sample of lan- 
guage may be divided into at least two 
classes of words, (1) the structural words 
and (2) the content words. It is possible, 
though less practicable, to give each 
word a weight in accordance with its 
classification in the above terms, or in 
accordance with its frequency of use in 
some standard language sample, and 
thus derive a more suitable measure of 
language. In any event, the procedure 
used will, to some degree, affect the 
character of the results obtained. 

The classification of word units ac- 


cording to certain rules and the count- 
ing of the resulting classes results in a 
measure of the number of types, or what 
may be appropriately called the vocab- 
ulary of the sample. Here again we note 
that our measure is a Statistical one, in 
that it involves classification and enu- 
meration. Once more, each type is given 
an equal weight in arriving at the nu- 
merical value of the measure, regardless 
of the function each type plays in the 
language structure. In this instance, how- 
ever, greater liberty is given us in setting 
up our classes. What is psychologically 
the more fruitful method of selecting 
these classes can only be speculated 
upon. It might be argued that ‘fall’, 


‘fall in’, ‘fall out’, ‘fall short’, ‘fall apart’ 


should each be classified as a unit rather 
than as two units on the grounds that 
each represents a unitary symbol. Such 
an argument is undoubtedly valid, and 
if such a procedure were followed it 
might conceivably alter the results. 

The language measures used in this 
study represent but a fraction. of those 
already in use or that have been sug- 
gested for use. Sanford (11) in a recently 
published investigation demonstrated 
the utility of some 234 language meas- 
ures which he used to describe person- 
ality differences between two individuals 
whose language he investigated. Buse- 
mann (2) has suggested and used the ad- 
jective-verb quotient for the study of 
personality. Boder (1), following Buse- 
mann’s suggestion, has also employed 
the adjective-verb quotient, this time in 
the study of various types of literature. 
Mann (9) also applied this measure in 
comparing schizophrenic patients with 
university freshmen; she also employed 
an adjective-noun quotient and an ad- 
verb-verb quotient. Johnson (7) has sug- 
gested several language measures, among 
which are included the ones used in this 
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study. Measures of language, not in 
terms of word counts, but in terms of 
themes and topics have been investigated 
by Skinner and his co-workers (12). This 
enumeration will give the reader at least 
a suggestion of the abundance of lan- 
guage measures already in use or pro- 
posed. Many others, of course, are pos- 
sibie. The ones used in the present study 
by no means exhaust the possibilities of 
language analysis. 

Further qualifications of the present 
results arise with regard to the fact that 
pertinent determining factors may have 
been omitted from consideration. ‘Two 
such factors may be (1) number of topics 
discussed by each individual and (2) rate 
of verbal output per unit of time. With 
regard to the first of the above factors, if 
we are permitted to assume that, other 
conditions being equal, the number of 
types in a sample of language is posi- 
tively correlated to the number of topics 
discussed in that sample, then it may 
appear plausible that, insofar as_ the 
more intelligent individuals can sustain 
a discussion on one topic for a greater 
number of words and, thus, for a speci- 
fied number of words discuss fewer topics 
than less intelligent individuals, who of 
a necessity must shift topics more fre- 
quently to write a given quota, the rela- 
tionship of intelligence to the number 
of types will be somewhat attenuated 
unless the number of topics is given 
some weight in the determination of this 
relationship. Again, it seems likely that 
age differences may be accentuated due 
to a wider range of interests, ambitions, 
opportunities, etc. of the older children. 
The behavior of these language measures 
within selected topics or fields of writ- 
ing, as for instance in fiction and scien- 
tific writing, may be profitably inves- 
tigated. In any event, consideration of 
this factor of number and type of topic, 


perhaps by an analysis of co-variance 
technique, will broaden our understand- 
ing of language in terms of these meas- 
ures. 

The second factor mentioned above, 
namely, rate of verbal output per unit 
of time, might be investigated by some- 
how weighting the number of types pro- 
duced by reference to the rate of verbal 
output. The analysis of co-variance tech- 
nique is an appropriate method of carry- 
ing out such a weighting. Fossum (6), for 
spoken language, reported a negative 
correlation of —.45 of 100-word type- 
token ratio with rate of verbal output 
per unit of time. Whether the relation- 
ship, if any, is in the same direction for 
written language is yet to be determined. 

These two above-mentioned factors do 
not exhaust all of the possibly pertinent 
factors which may need to be controlled, 
although they do offer perhaps the great- 
est promise of successful manipulation. 
Psychological factors such as motivation, 
interests, attitudes, ambitions, emotional 
States, etc. which are admittedly more 
difficult to control, nevertheless may 
prove to be significant determining fac- 
tors in differentiating individuals in 
terms of these language measures. It was 
noted that in some instances children 
who expressed a dislike for the task of 
writing 3,000 words seemed to show a 
tendency to write in short jerky sen- 
tences with many of the sentences be- 
ginning with the same pattern of words, 
producing a stereotyped effect. On the 
other hand, children who expressed in- 
terest and a liking for this task tended 
to keep their discussion varied through- 
out the manuscript and thus probably 
produced a greater number of types 
than they would have if the motivation 
had been less adequate. For example, 
the boy who produced the greatest num- 
ber of types in the present group of chil- 
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dren expressed a desire to be a writer 
of western fiction. Additional factors that 
may need to be given consideration in 
studying language in terms of these 
measures are: (1) differential effects of 
fatigue; (2) physiological conditions; (3) 
socio-economic background in the im- 
mediate family sense; (4) season of the 
year; (5) personality variables; etc. This 
enumeration certainly does not include 
all the possibly influential factors and 
serves only to demonstrate the complex- 
ity of the problem. 

One implication of the results of this 
study is that the segmental type-token 
ratios based on successively larger seg- 
ments are for practical purposes 
equivalent measures, insofar as they dif- 
ferentiate among individuals. This im- 
plication is inferred from the fact that 
the intercorrelation of these segmental 
type-token ratios are uniformly high and 
linear. However, there are instances in 
which the value of the type-token ratio 
for 100-word segments places individuals 
near the top of the group for this meas- 
ure while for these same individuals the 
value of the 3,000-word type-token ratio 
places them near the bottom of the 
group. Such individuals would seem to 
be making efficient use of the vocabulary 
available to them, and if such an inter- 
pretation is justified, development of the 
notion of efficiency of vocabulary usage 
should result in fruitful research. 

Certain implications arise from the 
curve-fitting aspects of the present study. 
In general, tests of the applicability of 
the equation presented by Carroll (3) 
point toward a rejection of the general- 
ity of this equation to these data. One 
possible reason for this may lie in the 
fact that the language samples used in 
this investigation were much different 
in certain characteristics from the lan- 
guage samples used by Carroll. The main 


points of difference are: (1) one type of 
language sample used by Carroll con- 
sisted of the verbal output of a group 
of subjects which was combined into one 
unit, while in this study each language 
sample represents the performance of 
but one child. It may well be that the 
aforementioned equation will hold for 
the first type of language sample but not 
for the second; (2) the other type of lan- 
guage sample used by Carroll was ob- 
tained from the field of literature, and 
the individuals who produced the writ- 
ing probably represented a highly se- 
lected group of verbally skilled individ- 
uals. In this study the verbal output of 
‘normal’ Iowa school children, who are 
comparatively unskilled in language arts, 
was the object of investigation; (3) great 
differences in age, intelligence and en- 
vironmental background undoubtedly 
existed between the two groups of sub- 
jects who produced the language studied 
in the two investigations. There is the 
possibility that, in general, the equation 
presented by Carroll will hold for some 
types of language samples but not for 
others. 

On the other hand, there may be some 
question as to whether the notion in- 
volved in this equation is too narrow for 
psychological utility. Although an equa- 
tion with only one parameter may be 
found to be adequate to describe the re- 
lationship between D and N, it would 
appear to be highly unlikely that the 
rate at which new words are added in 
connected discourse is so simply ex- 
plained. The factor of number of topics, 
for example, has already been discussed 
as a possible factor influencing the rate 
at which new words are added, and prob- 
ably other factors are also to be consid- 
ered. 


Attempts to fit a curve to represent the 
relationship between D and N reveals 
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that the function is of no simple nature. 
The study of the relationship between 
frequency of occurrence of a word and 
its rank demonstrates the important fact 
that the more frequently occurring 
words do not fit the same general func- 
tion as do the rarer words. Considera- 
tions of the language structure suggests 
that perhaps it may be feasible to divide 
the words of any language sample into 
two categories. First, there are the in- 
terstitial or structural words which form 
the core of framework structure of lan- 
guage. Since these words carry little 
meaning beyond the verbal context in 
which they appear they may be termed 
intensional words in contrast to the sec- 
ond type of words, the content and ac- 
tion words that have, directly or indi- 
rectly, an extensional reference. Since the 
more frequently occurring words are of 
the intensional type, we have a basis for 
making an analysis of the language into 
two parts. In view of the fact that the 
extensional words serve to represent or 
are symbolic of the interests, attitudes, 
ambitions, etc. of the writer they would 
appear to be of greater psychological in- 
terest. ‘There is some suggestion that if 
such a division of words could be made, 
a hyperbolic equation could be fitted to 
each division of words in the sample. 
One of the vitiating factors in attempts 
to find a lawful relationship between D 
and N may well lie in the difference be- 
tween the ways in which these two types 
of words reach their maximum. The in- 
tensional words appear to reach a maxi- 
mum in terms of D very rapidly, while 
the extensional words rise comparatively 
much more slowly and reach a maximum 
at a much later point on the curve. 


VI. SUMMARY AND CONCLUSIONS 


Three-thousand-word 


written _lan- 


guage samples were obtained from 108 
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Iowa school children who had been se- 
lected to fill the cells of a factorial design 
which consisted of three levels each of 
I.Q., C.A., and locality (city, town, rural) 
as well as two equal groups of boys and 
girls. The subjects were asked to write 
about whatever they wanted to write 
about and in a free-writing situation for 
a short time each day until they had 
reached their quota of 3,000 words. 

Each language sample was edited and 
the words tabulated according to a set 
of predetermined rules. From these tabu- 
lations, 20 language measures were ob- 
tained for each sample, and individual 
cumulative type-frequency curves were 
computed for a selected group of sub- 
jects. Language measures obtained were: 
(1) 100-word type-token ratio, (2) 500- 
word type-token ratio, (3) 1,000-word 
type-token ratio, (4) 3,000-word type- 
token ratio; number of tokens for (5) 
nounal, (6) verbal, (7) adjectival, (8) ad- 
verbal categories; number of types for 
(9) nounal, (10) verbal, (11) adjectival, 
(12) adverbal categories; type-token ratio 
for (13) nouns, (14) verbs, (15) adjectives, 
(16) adverbs; percentage of (17) nounal, 
(18) verbal, (19) adjectival, (20) adverbal 
types of the total types of these four parts 
of speech categories. 

These data were analyzed in three 
ways in order to determine (a) the re- 
liability of type-token ratios; (b) the 
ability of these measures to differentiate 
groups of individuals classified accord- 
ing to levels of I.Q., C.A., locality and 
sex; and (c) the mathematical relation- 
ship, if any, between the number of dif- 
ferent words (D) and the size of the sam- 
ple (N). 

On the basis of these analyses the fol- 
lowing conclusions can be drawn: 

1. Segmental type-token ratios derived 
from samples of 3,000 words are highly 
reliable in (a) the agreement between 





STUDIES IN LANGUAGE BEHAVIOR 111 


two independent sets of operations used 
to arrive at the numerical value of the 
iype-token ratio and in (b) the relative 
constancy of the type-token ratio over a 
short span of time (about a week). A fur- 
ther implication, indicated by the fact 
that the reliability coefficient is a positive 
function of the size of sample, is that, in 
general, type-token ratios computed 
from a sample of 1,000 words in length 
are, for practical purposes, as reliable as 
are those computed from samples of 
3,000 words in length and should prove 
satisfactory in all instances except when 
a high degree of precision is needed. 

2. The results of the analyses of vari- 
ance of the 20 language measures may be 
summarized briefly in the statement that 
the implication of these results is that 
the language measures employed can be 
used to characterize groups classified ac- 
cording to 1.Q., C.A., locality and pos- 
sibly sex, although the results for sex 
are practically negative. On the whole, 


the more highly developed the individ- 
ual in terms of intelligence and age, the 
more highly differentiated his language 
structure appears to be. This is shown 
particularly by the type-token ratios. It 
may also be said that high I.Q. groups 
are characterized by the use of a propor- 
tionately greater number of nouns while 
the low I.Q. groups are characterized by 
the use of a greater percentage of verbs 
and adverbs. 

g. An equation presented by Carroll 
was discussed and tests of its adequacy 
to describe these data were carried out. 
Its generality to these data was not sub- 
stantiated, although a more general form 
of the same equation was found to give 
a fairly good fit. The relationship of 
the number of different words to the size 
of sample was found to be a complex 
one. Empirically fitted equations to rep- 
resent these data are of such a character 
as to make prediction beyond the limits 
of the data hazardous, 
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ERRATA 


idfficulty should be difficulty. ‘aphs 


The following sentence should have appeared 
here as a footnote: A technique used by Dr. 
Hudson fost, Il. Inst. of Juvenile Research, 
Chicago, III. 
for should be on. 
divising should be devising. 

(Second Sample Made) 


Face is reddening. She chews her lips, then her 
fingers. She is breathing fast. Rubs her arms 
with palms of hands. Angry tone toward ob- 
server and flash-box is beginning to lessen. 
Voice is softer. Tears in eyes. 


ration 


(Third Sample Made) 


Face very red. Voice has become soft. Man- 
ner is apologetic. She is still trying. Lips are 
trembling. Voice a whisper, hesitating. Face 
has frightened expression. She mumbles, “I 
don’t know, I don’t know,” and “I can’t get it.” 
Lips trembling greatly. She frowns, at point of 
crying aloud, “I can’t get it. Oh, I never do any 
good!” Departs without a word, Silently shuts 
the door. 

socre should be score. 


groups should be group. 
his should be this. 


word of should be omitted. 

measured should be measure. 

who should be which. 

Insert test after intelligence. 
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